https://github.com/AgentOps-AI/tokencost Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} AgentOps-AI / tokencost Public * Notifications You must be signed in to change notification settings * Fork 16 * Star 303 * Easy token price estimates for 400+ LLMs agentops.ai License MIT license 303 stars 16 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 4 * Pull requests 2 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights AgentOps-AI/tokencost This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 169 Commits .github/workflows .github/ workflows tests tests tokencost tokencost .gitignore .gitignore LICENSE LICENSE MANIFEST.in MANIFEST.in README.md README.md pricing_table.md pricing_table.md pyproject.toml pyproject.toml tokencost.png tokencost.png tox.ini tox.ini update_prices.py update_prices.py View all files Repository files navigation * README * MIT license Tokencost Clientside token counting + price estimation for LLM apps and AI agents. Python Version Twitter * Discord * [?] AgentOps TokenCost License: MIT PyPI - Version X (formerly Twitter) Follow Tokencost helps calculate the USD cost of using major Large Language Model (LLMs) APIs by calculating the estimated cost of prompts and completions. Building AI agents? Check out AgentOps Features * LLM Price Tracking Major LLM providers frequently add new models and update pricing. This repo helps track the latest price changes * Token counting Accurately count prompt tokens before sending OpenAI requests * Easy integration Get the cost of a prompt or completion with a single function Example usage: from tokencost import calculate_prompt_cost, calculate_completion_cost model = "gpt-3.5-turbo" prompt = [{ "role": "user", "content": "Hello world"}] completion = "How may I assist you today?" prompt_cost = calculate_prompt_cost(prompt, model) completion_cost = calculate_completion_cost(completion, model) print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}") # 0.0000135 + 0.000014 = 0.0000275 Installation Recommended: PyPI: pip install tokencost Usage Cost estimates Calculating the cost of prompts and completions from OpenAI requests from openai import OpenAI client = OpenAI() model = "gpt-3.5-turbo" prompt = [{ "role": "user", "content": "Say this is a test"}] chat_completion = client.chat.completions.create( messages=prompt, model=model ) completion = chat_completion.choices[0].message.content # "This is a test." prompt_cost = calculate_prompt_cost(prompt, model) completion_cost = calculate_completion_cost(completion, model) print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}") # 0.0000180 + 0.000010 = 0.0000280 Calculating cost using string prompts instead of messages: from tokencost import calculate_prompt_cost prompt_string = "Hello world" response = "How may I assist you today?" model= "gpt-3.5-turbo" prompt_cost = calculate_prompt_cost(prompt_string, model) print(f"Cost: ${prompt_cost}") # Cost: $3e-06 Counting tokens from tokencost import count_message_tokens, count_string_tokens message_prompt = [{ "role": "user", "content": "Hello world"}] # Counting tokens in prompts formatted as message lists print(count_message_tokens(message_prompt, model="gpt-3.5-turbo")) # 9 # Alternatively, counting tokens in string prompts print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo")) # 2 Cost table Units denominated in USD. All prices can be located in model_prices.json. * Prices last updated Jan 30, 2024 from LiteLLM's cost dictionary Prompt Cost Completion Max Max Model Name (USD) Cost (USD) Prompt Output Tokens Tokens gpt-4 $0.00003000 $0.00006000 8,192 4096 gpt-4o $0.00000500 $0.00001500 128,000 4096 gpt-4o-2024-05-13 $0.00000500 $0.00001500 128,000 4096 gpt-4-turbo-preview $0.00001000 $0.00003000 128,000 4096 gpt-4-0314 $0.00003000 $0.00006000 8,192 4096 gpt-4-0613 $0.00003000 $0.00006000 8,192 4096 gpt-4-32k $0.00006000 $0.00012000 32,768 4096 gpt-4-32k-0314 $0.00006000 $0.00012000 32,768 4096 gpt-4-32k-0613 $0.00006000 $0.00012000 32,768 4096 gpt-4-turbo $0.00001000 $0.00003000 128,000 4096 gpt-4-turbo-2024-04-09 $0.00001000 $0.00003000 128,000 4096 gpt-4-1106-preview $0.00001000 $0.00003000 128,000 4096 gpt-4-0125-preview $0.00001000 $0.00003000 128,000 4096 gpt-4-vision-preview $0.00001000 $0.00003000 128,000 4096 gpt-4-1106-vision-preview $0.00001000 $0.00003000 128,000 4096 gpt-3.5-turbo $0.00000150 $0.00000200 16,385 4096 gpt-3.5-turbo-0301 $0.00000150 $0.00000200 4,097 4096 gpt-3.5-turbo-0613 $0.00000150 $0.00000200 4,097 4096 gpt-3.5-turbo-1106 $0.00000100 $0.00000200 16,385 4096 gpt-3.5-turbo-0125 $0.00000050 $0.00000150 16,385 4096 gpt-3.5-turbo-16k $0.00000300 $0.00000400 16,385 4096 gpt-3.5-turbo-16k-0613 $0.00000300 $0.00000400 16,385 4096 ft:gpt-3.5-turbo $0.00000300 $0.00000600 4,097 4096 ft:davinci-002 $0.00000200 $0.00000200 16,384 4096 ft:babbage-002 $0.00000040 $0.00000040 16,384 4096 text-embedding-3-large $0.00000013 $0.00000000 8,191 nan text-embedding-3-small $0.00000002 $0.00000000 8,191 nan text-embedding-ada-002 $0.00000010 $0.00000000 8,191 nan text-embedding-ada-002-v2 $0.00000010 $0.00000000 8,191 nan text-moderation-stable $0.00000000 $0.00000000 32,768 0 text-moderation-007 $0.00000000 $0.00000000 32,768 0 text-moderation-latest $0.00000000 $0.00000000 32,768 0 256-x-256/dall-e-2 -- -- nan nan 512-x-512/dall-e-2 -- -- nan nan 1024-x-1024/dall-e-2 -- -- nan nan hd/1024-x-1792/dall-e-3 -- -- nan nan hd/1792-x-1024/dall-e-3 -- -- nan nan hd/1024-x-1024/dall-e-3 -- -- nan nan standard/1024-x-1792/dall-e-3 -- -- nan nan standard/1792-x-1024/dall-e-3 -- -- nan nan standard/1024-x-1024/dall-e-3 -- -- nan nan whisper-1 -- -- nan nan azure/whisper-1 -- -- nan nan azure/gpt-4o $0.00000500 $0.00001500 128,000 4096 azure/gpt-4-turbo-2024-04-09 $0.00001000 $0.00003000 128,000 4096 azure/gpt-4-0125-preview $0.00001000 $0.00003000 128,000 4096 azure/gpt-4-1106-preview $0.00001000 $0.00003000 128,000 4096 azure/gpt-4-0613 $0.00003000 $0.00006000 8,192 4096 azure/gpt-4-32k-0613 $0.00006000 $0.00012000 32,768 4096 azure/gpt-4-32k $0.00006000 $0.00012000 32,768 4096 azure/gpt-4 $0.00003000 $0.00006000 8,192 4096 azure/gpt-4-turbo $0.00001000 $0.00003000 128,000 4096 azure/gpt-4-turbo-vision-preview $0.00001000 $0.00003000 128,000 4096 azure/gpt-35-turbo-16k-0613 $0.00000300 $0.00000400 16,385 4096 azure/gpt-35-turbo-1106 $0.00000150 $0.00000200 16,384 4096 azure/gpt-35-turbo-0125 $0.00000050 $0.00000150 16,384 4096 azure/gpt-35-turbo-16k $0.00000300 $0.00000400 16,385 4096 azure/gpt-35-turbo $0.00000050 $0.00000150 4,097 4096 azure/gpt-3.5-turbo-instruct-0914 $0.00000150 $0.00000200 4,097 nan azure/gpt-35-turbo-instruct $0.00000150 $0.00000200 4,097 nan azure/mistral-large-latest $0.00000800 $0.00002400 32,000 nan azure/mistral-large-2402 $0.00000800 $0.00002400 32,000 nan azure/command-r-plus $0.00000300 $0.00001500 128,000 4096 azure/ada $0.00000010 $0.00000000 8,191 nan azure/text-embedding-ada-002 $0.00000010 $0.00000000 8,191 nan azure/text-embedding-3-large $0.00000013 $0.00000000 8,191 nan azure/text-embedding-3-small $0.00000002 $0.00000000 8,191 nan azure/standard/1024-x-1024/dall-e-3 -- $0.00000000 nan nan azure/hd/1024-x-1024/dall-e-3 -- $0.00000000 nan nan azure/standard/1024-x-1792/dall-e-3 -- $0.00000000 nan nan azure/standard/1792-x-1024/dall-e-3 -- $0.00000000 nan nan azure/hd/1024-x-1792/dall-e-3 -- $0.00000000 nan nan azure/hd/1792-x-1024/dall-e-3 -- $0.00000000 nan nan azure/standard/1024-x-1024/dall-e-2 -- $0.00000000 nan nan babbage-002 $0.00000040 $0.00000040 16,384 4096 davinci-002 $0.00000200 $0.00000200 16,384 4096 gpt-3.5-turbo-instruct $0.00000150 $0.00000200 8,192 4096 gpt-3.5-turbo-instruct-0914 $0.00000150 $0.00000200 8,192 4097 claude-instant-1 $0.00000163 $0.00000551 100,000 8191 mistral/mistral-tiny $0.00000025 $0.00000025 32,000 8191 mistral/mistral-small $0.00000100 $0.00000300 32,000 8191 mistral/mistral-small-latest $0.00000100 $0.00000300 32,000 8191 mistral/mistral-medium $0.00000270 $0.00000810 32,000 8191 mistral/mistral-medium-latest $0.00000270 $0.00000810 32,000 8191 mistral/mistral-medium-2312 $0.00000270 $0.00000810 32,000 8191 mistral/mistral-large-latest $0.00000400 $0.00001200 32,000 8191 mistral/mistral-large-2402 $0.00000400 $0.00001200 32,000 8191 mistral/open-mistral-7b $0.00000025 $0.00000025 32,000 8191 mistral/open-mixtral-8x7b $0.00000070 $0.00000070 32,000 8191 mistral/open-mixtral-8x22b $0.00000200 $0.00000600 64,000 8191 mistral/codestral-latest $0.00000100 $0.00000300 32,000 8191 mistral/codestral-2405 $0.00000100 $0.00000300 32,000 8191 mistral/mistral-embed $0.00000010 -- 8,192 nan deepseek-chat $0.00000014 $0.00000028 32,000 4096 deepseek-coder $0.00000014 $0.00000028 16,000 4096 groq/llama2-70b-4096 $0.00000070 $0.00000080 4,096 4096 groq/llama3-8b-8192 $0.00000010 $0.00000010 8,192 8192 groq/llama3-70b-8192 $0.00000064 $0.00000080 8,192 8192 groq/mixtral-8x7b-32768 $0.00000027 $0.00000027 32,768 32768 groq/gemma-7b-it $0.00000010 $0.00000010 8,192 8192 claude-instant-1.2 $0.00000016 $0.00000055 100,000 8191 claude-2 $0.00000800 $0.00002400 100,000 8191 claude-2.1 $0.00000800 $0.00002400 200,000 8191 claude-3-haiku-20240307 $0.00000025 $0.00000125 200,000 4096 claude-3-opus-20240229 $0.00001500 $0.00007500 200,000 4096 claude-3-sonnet-20240229 $0.00000300 $0.00001500 200,000 4096 text-bison $0.00000012 $0.00000012 8,192 1024 text-bison@001 $0.00000012 $0.00000012 8,192 1024 text-unicorn $0.00001000 $0.00002800 8,192 1024 text-unicorn@001 $0.00001000 $0.00002800 8,192 1024 chat-bison $0.00000012 $0.00000012 8,192 4096 chat-bison@001 $0.00000012 $0.00000012 8,192 4096 chat-bison@002 $0.00000012 $0.00000012 8,192 4096 chat-bison-32k $0.00000012 $0.00000012 32,000 8192 code-bison $0.00000012 $0.00000012 6,144 1024 code-bison@001 $0.00000012 $0.00000012 6,144 1024 code-gecko@001 $0.00000012 $0.00000012 2,048 64 code-gecko@002 $0.00000012 $0.00000012 2,048 64 code-gecko $0.00000012 $0.00000012 2,048 64 codechat-bison $0.00000012 $0.00000012 6,144 1024 codechat-bison@001 $0.00000012 $0.00000012 6,144 1024 codechat-bison-32k $0.00000012 $0.00000012 32,000 8192 gemini-pro $0.00000025 $0.00000050 32,760 8192 gemini-1.0-pro $0.00000025 $0.00000050 32,760 8192 gemini-1.0-pro-001 $0.00000025 $0.00000050 32,760 8192 gemini-1.0-pro-002 $0.00000025 $0.00000050 32,760 8192 gemini-1.5-pro $0.00000063 $0.00000188 1,000,000 8192 gemini-1.5-flash-001 $0.00000000 $0.00000000 1,000,000 8192 gemini-1.5-flash-preview-0514 $0.00000000 $0.00000000 1,000,000 8192 gemini-1.5-pro-001 $0.00000063 $0.00000188 1,000,000 8192 gemini-1.5-pro-preview-0514 $0.00000063 $0.00000188 1,000,000 8192 gemini-1.5-pro-preview-0215 $0.00000063 $0.00000188 1,000,000 8192 gemini-1.5-pro-preview-0409 $0.00000063 $0.00000188 1,000,000 8192 gemini-experimental $0.00000000 $0.00000000 1,000,000 8192 gemini-pro-vision $0.00000025 $0.00000050 16,384 2048 gemini-1.0-pro-vision $0.00000025 $0.00000050 16,384 2048 gemini-1.0-pro-vision-001 $0.00000025 $0.00000050 16,384 2048 vertex_ai/claude-3-sonnet@20240229 $0.00000300 $0.00001500 200,000 4096 vertex_ai/claude-3-haiku@20240307 $0.00000025 $0.00000125 200,000 4096 vertex_ai/claude-3-opus@20240229 $0.00001500 $0.00007500 200,000 4096 vertex_ai/imagegeneration@006 -- -- nan nan textembedding-gecko $0.00000001 $0.00000000 3,072 nan textembedding-gecko-multilingual $0.00000001 $0.00000000 3,072 nan textembedding-gecko-multilingual@001 $0.00000001 $0.00000000 3,072 nan textembedding-gecko@001 $0.00000001 $0.00000000 3,072 nan textembedding-gecko@003 $0.00000001 $0.00000000 3,072 nan text-embedding-preview-0409 $0.00000001 $0.00000000 3,072 nan text-multilingual-embedding-preview-0409 $0.00000001 $0.00000000 3,072 nan palm/chat-bison $0.00000012 $0.00000012 8,192 4096 palm/chat-bison-001 $0.00000012 $0.00000012 8,192 4096 palm/text-bison $0.00000012 $0.00000012 8,192 1024 palm/text-bison-001 $0.00000012 $0.00000012 8,192 1024 palm/text-bison-safety-off $0.00000012 $0.00000012 8,192 1024 palm/text-bison-safety-recitation-off $0.00000012 $0.00000012 8,192 1024 gemini/gemini-1.5-flash-latest $0.00000000 $0.00000000 1,000,000 8192 gemini/gemini-pro $0.00000000 $0.00000000 32,760 8192 gemini/gemini-1.5-pro $0.00000000 $0.00000000 1,000,000 8192 gemini/gemini-1.5-pro-latest $0.00000000 $0.00000000 1,048,576 8192 gemini/gemini-pro-vision $0.00000000 $0.00000000 30,720 2048 command-r $0.00000050 $0.00000150 128,000 4096 command-light $0.00001500 $0.00001500 4,096 4096 command-r-plus $0.00000300 $0.00001500 128,000 4096 command-nightly $0.00001500 $0.00001500 4,096 4096 command $0.00001500 $0.00001500 4,096 4096 command-medium-beta $0.00001500 $0.00001500 4,096 4096 command-xlarge-beta $0.00001500 $0.00001500 4,096 4096 replicate/meta/llama-2-13b $0.00000010 $0.00000050 4,096 4096 replicate/meta/llama-2-13b-chat $0.00000010 $0.00000050 4,096 4096 replicate/meta/llama-2-70b $0.00000065 $0.00000275 4,096 4096 replicate/meta/llama-2-70b-chat $0.00000065 $0.00000275 4,096 4096 replicate/meta/llama-2-7b $0.00000005 $0.00000025 4,096 4096 replicate/meta/llama-2-7b-chat $0.00000005 $0.00000025 4,096 4096 replicate/meta/llama-3-70b $0.00000065 $0.00000275 8,192 8192 replicate/meta/llama-3-70b-instruct $0.00000065 $0.00000275 8,192 8192 replicate/meta/llama-3-8b $0.00000005 $0.00000025 8,086 8086 replicate/meta/llama-3-8b-instruct $0.00000005 $0.00000025 8,086 8086 replicate/mistralai/mistral-7b-v0.1 $0.00000005 $0.00000025 4,096 4096 replicate/mistralai/ $0.00000005 $0.00000025 4,096 4096 mistral-7b-instruct-v0.2 replicate/mistralai/ $0.00000030 $0.00000100 4,096 4096 mixtral-8x7b-instruct-v0.1 openrouter/microsoft/ $0.00000100 $0.00000100 nan nan wizardlm-2-8x22b:nitro openrouter/google/gemini-pro-1.5 $0.00000250 $0.00000750 1,000,000 8192 openrouter/mistralai/ $0.00000065 $0.00000065 nan nan mixtral-8x22b-instruct openrouter/cohere/command-r-plus $0.00000300 $0.00001500 nan nan openrouter/databricks/dbrx-instruct $0.00000060 $0.00000060 nan nan openrouter/anthropic/claude-3-haiku $0.00000025 $0.00000125 nan nan openrouter/anthropic/claude-3-sonnet $0.00000300 $0.00001500 nan nan openrouter/mistralai/mistral-large $0.00000800 $0.00002400 nan nan openrouter/cognitivecomputations/ $0.00000050 $0.00000050 nan nan dolphin-mixtral-8x7b openrouter/google/gemini-pro-vision $0.00000012 $0.00000038 nan nan openrouter/fireworks/firellava-13b $0.00000020 $0.00000020 nan nan openrouter/meta-llama/ $0.00000000 $0.00000000 nan nan llama-3-8b-instruct:free openrouter/meta-llama/ $0.00000022 $0.00000225 nan nan llama-3-8b-instruct:extended openrouter/meta-llama/ $0.00000090 $0.00000090 nan nan llama-3-70b-instruct:nitro openrouter/meta-llama/ $0.00000059 $0.00000079 nan nan llama-3-70b-instruct openrouter/openai/gpt-4o $0.00000500 $0.00001500 128,000 4096 openrouter/openai/gpt-4o-2024-05-13 $0.00000500 $0.00001500 128,000 4096 openrouter/openai/gpt-4-vision-preview $0.00001000 $0.00003000 nan nan openrouter/openai/gpt-3.5-turbo $0.00000150 $0.00000200 nan nan openrouter/openai/gpt-3.5-turbo-16k $0.00000300 $0.00000400 nan nan openrouter/openai/gpt-4 $0.00003000 $0.00006000 nan nan openrouter/anthropic/claude-instant-v1 $0.00000163 $0.00000551 nan 8191 openrouter/anthropic/claude-2 $0.00001102 $0.00003268 nan 8191 openrouter/anthropic/claude-3-opus $0.00001500 $0.00007500 200,000 4096 openrouter/google/palm-2-chat-bison $0.00000050 $0.00000050 nan nan openrouter/google/palm-2-codechat-bison $0.00000050 $0.00000050 nan nan openrouter/meta-llama/llama-2-13b-chat $0.00000020 $0.00000020 nan nan openrouter/meta-llama/llama-2-70b-chat $0.00000150 $0.00000150 nan nan openrouter/meta-llama/ $0.00000050 $0.00000050 nan nan codellama-34b-instruct openrouter/nousresearch/ $0.00000020 $0.00000020 nan nan nous-hermes-llama2-13b openrouter/mancer/weaver $0.00000563 $0.00000563 nan nan openrouter/gryphe/mythomax-l2-13b $0.00000188 $0.00000188 nan nan openrouter/jondurbin/ $0.00001388 $0.00001388 nan nan airoboros-l2-70b-2.1 openrouter/undi95/remm-slerp-l2-13b $0.00000188 $0.00000188 nan nan openrouter/pygmalionai/mythalion-13b $0.00000188 $0.00000188 nan nan openrouter/mistralai/mistral-7b-instruct $0.00000013 $0.00000013 nan nan openrouter/mistralai/ $0.00000000 $0.00000000 nan nan mistral-7b-instruct:free j2-ultra $0.00001500 $0.00001500 8,192 8192 j2-mid $0.00001000 $0.00001000 8,192 8192 j2-light $0.00000300 $0.00000300 8,192 8192 dolphin $0.00000050 $0.00000050 16,384 16384 chatdolphin $0.00000050 $0.00000050 16,384 16384 luminous-base $0.00003000 $0.00003300 nan nan luminous-base-control $0.00003750 $0.00004125 nan nan luminous-extended $0.00004500 $0.00004950 nan nan luminous-extended-control $0.00005625 $0.00006187 nan nan luminous-supreme $0.00017500 $0.00019250 nan nan luminous-supreme-control $0.00021875 $0.00024063 nan nan ai21.j2-mid-v1 $0.00001250 $0.00001250 8,191 8191 ai21.j2-ultra-v1 $0.00001880 $0.00001880 8,191 8191 amazon.titan-text-lite-v1 $0.00000030 $0.00000040 42,000 4000 amazon.titan-text-express-v1 $0.00000130 $0.00000170 42,000 8000 amazon.titan-embed-text-v1 $0.00000010 $0.00000000 8,192 nan amazon.titan-embed-text-v2:0 $0.00000020 $0.00000000 8,192 nan mistral.mistral-7b-instruct-v0:2 $0.00000015 $0.00000020 32,000 8191 mistral.mixtral-8x7b-instruct-v0:1 $0.00000045 $0.00000070 32,000 8191 mistral.mistral-large-2402-v1:0 $0.00000800 $0.00002400 32,000 8191 bedrock/us-west-2/ $0.00000045 $0.00000070 32,000 8191 mistral.mixtral-8x7b-instruct-v0:1 bedrock/us-east-1/ $0.00000045 $0.00000070 32,000 8191 mistral.mixtral-8x7b-instruct-v0:1 bedrock/eu-west-3/ $0.00000059 $0.00000091 32,000 8191 mistral.mixtral-8x7b-instruct-v0:1 bedrock/us-west-2/ $0.00000015 $0.00000020 32,000 8191 mistral.mistral-7b-instruct-v0:2 bedrock/us-east-1/ $0.00000015 $0.00000020 32,000 8191 mistral.mistral-7b-instruct-v0:2 bedrock/eu-west-3/ $0.00000020 $0.00000026 32,000 8191 mistral.mistral-7b-instruct-v0:2 bedrock/us-east-1/ $0.00000800 $0.00002400 32,000 8191 mistral.mistral-large-2402-v1:0 bedrock/us-west-2/ $0.00000800 $0.00002400 32,000 8191 mistral.mistral-large-2402-v1:0 bedrock/eu-west-3/ $0.00001040 $0.00003120 32,000 8191 mistral.mistral-large-2402-v1:0 anthropic.claude-3-sonnet-20240229-v1:0 $0.00000300 $0.00001500 200,000 4096 anthropic.claude-3-haiku-20240307-v1:0 $0.00000025 $0.00000125 200,000 4096 anthropic.claude-3-opus-20240229-v1:0 $0.00001500 $0.00007500 200,000 4096 anthropic.claude-v1 $0.00000800 $0.00002400 100,000 8191 bedrock/us-east-1/anthropic.claude-v1 $0.00000800 $0.00002400 100,000 8191 bedrock/us-west-2/anthropic.claude-v1 $0.00000800 $0.00002400 100,000 8191 bedrock/ap-northeast-1/ $0.00000800 $0.00002400 100,000 8191 anthropic.claude-v1 bedrock/ap-northeast-1/ -- -- 100,000 8191 1-month-commitment/anthropic.claude-v1 bedrock/ap-northeast-1/ -- -- 100,000 8191 6-month-commitment/anthropic.claude-v1 bedrock/eu-central-1/anthropic.claude-v1 $0.00000800 $0.00002400 100,000 8191 bedrock/eu-central-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 bedrock/eu-central-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 bedrock/us-east-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 bedrock/us-east-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 bedrock/us-west-2/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 bedrock/us-west-2/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v1 anthropic.claude-v2 $0.00000800 $0.00002400 100,000 8191 bedrock/us-east-1/anthropic.claude-v2 $0.00000800 $0.00002400 100,000 8191 bedrock/us-west-2/anthropic.claude-v2 $0.00000800 $0.00002400 100,000 8191 bedrock/ap-northeast-1/ $0.00000800 $0.00002400 100,000 8191 anthropic.claude-v2 bedrock/ap-northeast-1/ -- -- 100,000 8191 1-month-commitment/anthropic.claude-v2 bedrock/ap-northeast-1/ -- -- 100,000 8191 6-month-commitment/anthropic.claude-v2 bedrock/eu-central-1/anthropic.claude-v2 $0.00000800 $0.00002400 100,000 8191 bedrock/eu-central-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 bedrock/eu-central-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 bedrock/us-east-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 bedrock/us-east-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 bedrock/us-west-2/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 bedrock/us-west-2/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2 anthropic.claude-v2:1 $0.00000800 $0.00002400 100,000 8191 bedrock/us-east-1/anthropic.claude-v2:1 $0.00000800 $0.00002400 100,000 8191 bedrock/us-west-2/anthropic.claude-v2:1 $0.00000800 $0.00002400 100,000 8191 bedrock/ap-northeast-1/ $0.00000800 $0.00002400 100,000 8191 anthropic.claude-v2:1 bedrock/ap-northeast-1/ -- -- 100,000 8191 1-month-commitment/anthropic.claude-v2:1 bedrock/ap-northeast-1/ -- -- 100,000 8191 6-month-commitment/anthropic.claude-v2:1 bedrock/eu-central-1/ $0.00000800 $0.00002400 100,000 8191 anthropic.claude-v2:1 bedrock/eu-central-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 bedrock/eu-central-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 bedrock/us-east-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 bedrock/us-east-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 bedrock/us-west-2/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 bedrock/us-west-2/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-v2:1 anthropic.claude-instant-v1 $0.00000163 $0.00000551 100,000 8191 bedrock/us-east-1/ $0.00000080 $0.00000240 100,000 8191 anthropic.claude-instant-v1 bedrock/us-east-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/us-east-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/us-west-2/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/us-west-2/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/us-west-2/ $0.00000080 $0.00000240 100,000 8191 anthropic.claude-instant-v1 bedrock/ap-northeast-1/ $0.00000223 $0.00000755 100,000 8191 anthropic.claude-instant-v1 bedrock/ap-northeast-1/ 1-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/ap-northeast-1/ 6-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/eu-central-1/ $0.00000248 $0.00000838 100,000 8191 anthropic.claude-instant-v1 bedrock/eu-central-1/1-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 bedrock/eu-central-1/6-month-commitment/ -- -- 100,000 8191 anthropic.claude-instant-v1 cohere.command-text-v14 $0.00000150 $0.00000200 4,096 4096 bedrock/*/1-month-commitment/ -- -- 4,096 4096 cohere.command-text-v14 bedrock/*/6-month-commitment/ -- -- 4,096 4096 cohere.command-text-v14 cohere.command-light-text-v14 $0.00000030 $0.00000060 4,096 4096 bedrock/*/1-month-commitment/ -- -- 4,096 4096 cohere.command-light-text-v14 bedrock/*/6-month-commitment/ -- -- 4,096 4096 cohere.command-light-text-v14 cohere.command-r-plus-v1:0 $0.00000300 $0.00001500 128,000 4096 cohere.command-r-v1:0 $0.00000050 $0.00000150 128,000 4096 cohere.embed-english-v3 $0.00000010 $0.00000000 512 nan cohere.embed-multilingual-v3 $0.00000010 $0.00000000 512 nan meta.llama2-13b-chat-v1 $0.00000075 $0.00000100 4,096 4096 meta.llama2-70b-chat-v1 $0.00000195 $0.00000256 4,096 4096 meta.llama3-8b-instruct-v1:0 $0.00000040 $0.00000060 8,192 8192 meta.llama3-70b-instruct-v1:0 $0.00000265 $0.00000350 8,192 8192 512-x-512/50-steps/ -- -- 77 nan stability.stable-diffusion-xl-v0 512-x-512/max-steps/ -- -- 77 nan stability.stable-diffusion-xl-v0 max-x-max/50-steps/ -- -- 77 nan stability.stable-diffusion-xl-v0 max-x-max/max-steps/ -- -- 77 nan stability.stable-diffusion-xl-v0 1024-x-1024/50-steps/ -- -- 77 nan stability.stable-diffusion-xl-v1 1024-x-1024/max-steps/ -- -- 77 nan stability.stable-diffusion-xl-v1 sagemaker/meta-textgeneration-llama-2-7b $0.00000000 $0.00000000 4,096 4096 sagemaker/ $0.00000000 $0.00000000 4,096 4096 meta-textgeneration-llama-2-7b-f sagemaker/ $0.00000000 $0.00000000 4,096 4096 meta-textgeneration-llama-2-13b sagemaker/ $0.00000000 $0.00000000 4,096 4096 meta-textgeneration-llama-2-13b-f sagemaker/ $0.00000000 $0.00000000 4,096 4096 meta-textgeneration-llama-2-70b sagemaker/ $0.00000000 $0.00000000 4,096 4096 meta-textgeneration-llama-2-70b-b-f together-ai-up-to-3b $0.00000010 $0.00000010 nan nan together-ai-3.1b-7b $0.00000020 $0.00000020 nan nan together-ai-7.1b-20b $0.00000040 $0.00000040 nan nan together-ai-20.1b-40b $0.00000080 $0.00000080 nan nan together-ai-40.1b-70b $0.00000090 $0.00000090 nan nan together_ai/mistralai/ $0.00000060 $0.00000060 nan nan Mixtral-8x7B-Instruct-v0.1 together_ai/mistralai/ -- -- nan nan Mistral-7B-Instruct-v0.1 together_ai/togethercomputer/ -- -- nan nan CodeLlama-34b-Instruct ollama/llama2 $0.00000000 $0.00000000 4,096 4096 ollama/llama2:13b $0.00000000 $0.00000000 4,096 4096 ollama/llama2:70b $0.00000000 $0.00000000 4,096 4096 ollama/llama2-uncensored $0.00000000 $0.00000000 4,096 4096 ollama/llama3 $0.00000000 $0.00000000 8,192 8192 ollama/llama3:70b $0.00000000 $0.00000000 8,192 8192 ollama/mistral $0.00000000 $0.00000000 8,192 8192 ollama/mistral-7B-Instruct-v0.1 $0.00000000 $0.00000000 8,192 8192 ollama/mistral-7B-Instruct-v0.2 $0.00000000 $0.00000000 32,768 32768 ollama/mixtral-8x7B-Instruct-v0.1 $0.00000000 $0.00000000 32,768 32768 ollama/mixtral-8x22B-Instruct-v0.1 $0.00000000 $0.00000000 65,536 65536 ollama/codellama $0.00000000 $0.00000000 4,096 4096 ollama/orca-mini $0.00000000 $0.00000000 4,096 4096 ollama/vicuna $0.00000000 $0.00000000 2,048 2048 deepinfra/lizpreciatior/lzlv_70b_fp16_hf $0.00000070 $0.00000090 4,096 4096 deepinfra/Gryphe/MythoMax-L2-13b $0.00000022 $0.00000022 4,096 4096 deepinfra/mistralai/ $0.00000013 $0.00000013 32,768 8191 Mistral-7B-Instruct-v0.1 deepinfra/meta-llama/Llama-2-70b-chat-hf $0.00000070 $0.00000090 4,096 4096 deepinfra/cognitivecomputations/ $0.00000027 $0.00000027 32,768 8191 dolphin-2.6-mixtral-8x7b deepinfra/codellama/ $0.00000060 $0.00000060 4,096 4096 CodeLlama-34b-Instruct-hf deepinfra/deepinfra/mixtral $0.00000027 $0.00000027 32,000 4096 deepinfra/Phind/Phind-CodeLlama-34B-v2 $0.00000060 $0.00000060 16,384 4096 deepinfra/mistralai/ $0.00000027 $0.00000027 32,768 8191 Mixtral-8x7B-Instruct-v0.1 deepinfra/deepinfra/airoboros-70b $0.00000070 $0.00000090 4,096 4096 deepinfra/01-ai/Yi-34B-Chat $0.00000060 $0.00000060 4,096 4096 deepinfra/01-ai/Yi-6B-200K $0.00000013 $0.00000013 200,000 4096 deepinfra/jondurbin/ $0.00000070 $0.00000090 4,096 4096 airoboros-l2-70b-gpt4-1.4.1 deepinfra/meta-llama/Llama-2-13b-chat-hf $0.00000022 $0.00000022 4,096 4096 deepinfra/amazon/MistralLite $0.00000020 $0.00000020 32,768 8191 deepinfra/meta-llama/Llama-2-7b-chat-hf $0.00000013 $0.00000013 4,096 4096 deepinfra/01-ai/Yi-34B-200K $0.00000060 $0.00000060 200,000 4096 deepinfra/openchat/openchat_3.5 $0.00000013 $0.00000013 4,096 4096 perplexity/codellama-34b-instruct $0.00000035 $0.00000140 16,384 16384 perplexity/codellama-70b-instruct $0.00000070 $0.00000280 16,384 16384 perplexity/pplx-7b-chat $0.00000007 $0.00000028 8,192 8192 perplexity/pplx-70b-chat $0.00000070 $0.00000280 4,096 4096 perplexity/pplx-7b-online $0.00000000 $0.00000028 4,096 4096 perplexity/pplx-70b-online $0.00000000 $0.00000280 4,096 4096 perplexity/llama-2-70b-chat $0.00000070 $0.00000280 4,096 4096 perplexity/mistral-7b-instruct $0.00000007 $0.00000028 4,096 4096 perplexity/mixtral-8x7b-instruct $0.00000007 $0.00000028 4,096 4096 perplexity/sonar-small-chat $0.00000007 $0.00000028 16,384 16384 perplexity/sonar-small-online $0.00000000 $0.00000028 12,000 12000 perplexity/sonar-medium-chat $0.00000060 $0.00000180 16,384 16384 perplexity/sonar-medium-online $0.00000000 $0.00000180 12,000 12000 anyscale/mistralai/ $0.00000015 $0.00000015 16,384 16384 Mistral-7B-Instruct-v0.1 anyscale/mistralai/ $0.00000015 $0.00000015 16,384 16384 Mixtral-8x7B-Instruct-v0.1 anyscale/mistralai/ $0.00000090 $0.00000090 65,536 65536 Mixtral-8x22B-Instruct-v0.1 anyscale/HuggingFaceH4/zephyr-7b-beta $0.00000015 $0.00000015 16,384 16384 anyscale/google/gemma-7b-it $0.00000015 $0.00000015 8,192 8192 anyscale/meta-llama/Llama-2-7b-chat-hf $0.00000015 $0.00000015 4,096 4096 anyscale/meta-llama/Llama-2-13b-chat-hf $0.00000025 $0.00000025 4,096 4096 anyscale/meta-llama/Llama-2-70b-chat-hf $0.00000100 $0.00000100 4,096 4096 anyscale/codellama/ $0.00000100 $0.00000100 4,096 4096 CodeLlama-34b-Instruct-hf anyscale/codellama/ $0.00000100 $0.00000100 4,096 4096 CodeLlama-70b-Instruct-hf anyscale/meta-llama/ $0.00000015 $0.00000015 8,192 8192 Meta-Llama-3-8B-Instruct anyscale/meta-llama/ $0.00000100 $0.00000100 8,192 8192 Meta-Llama-3-70B-Instruct cloudflare/@cf/meta/llama-2-7b-chat-fp16 $0.00000192 $0.00000192 3,072 3072 cloudflare/@cf/meta/llama-2-7b-chat-int8 $0.00000192 $0.00000192 2,048 2048 cloudflare/@cf/mistral/ $0.00000192 $0.00000192 8,192 8192 mistral-7b-instruct-v0.1 cloudflare/@hf/thebloke/ $0.00000192 $0.00000192 4,096 4096 codellama-7b-instruct-awq voyage/voyage-01 $0.00000010 $0.00000000 4,096 nan voyage/voyage-lite-01 $0.00000010 $0.00000000 4,096 nan voyage/voyage-large-2 $0.00000012 $0.00000000 16,000 nan voyage/voyage-law-2 $0.00000012 $0.00000000 16,000 nan voyage/voyage-code-2 $0.00000012 $0.00000000 16,000 nan voyage/voyage-2 $0.00000010 $0.00000000 4,000 nan voyage/voyage-lite-02-instruct $0.00000010 $0.00000000 4,000 nan databricks/databricks-dbrx-instruct $0.00000075 $0.00000225 32,768 32768 databricks/ $0.00000100 $0.00000300 8,192 8192 databricks-meta-llama-3-70b-instruct databricks/databricks-llama-2-70b-chat $0.00000050 $0.00000150 4,096 4096 databricks/ $0.00000050 $0.00000100 4,096 4096 databricks-mixtral-8x7b-instruct databricks/databricks-mpt-30b-instruct $0.00000100 $0.00000100 8,192 8192 databricks/databricks-mpt-7b-instruct $0.00000050 $0.00000050 8,192 8192 databricks/databricks-bge-large-en $0.00000010 $0.00000000 512 nan Callback handlers You may also calculate token costs in LLM wrapper/framework libraries using callbacks. LlamaIndex pip install `'tokencost[llama-index]'` To use the base callback handler, you may import it: from tokencost.callbacks.llama_index import TokenCostHandler and pass to your framework callback handler. Langchain (Coming Soon) Running locally Installation via GitHub: git clone git@github.com:AgentOps-AI/tokencost.git cd tokencost pip install -e . Running tests 0. Install pytest if you don't have it already pip install pytest 1. Run the tests/ folder while in the parent directory pytest tests This repo also supports tox, simply run python -m tox. Contributing Contributions to TokenCost are welcome! Feel free to create an issue for any bug reports, complaints, or feature suggestions. License TokenCost is released under the MIT License. About Easy token price estimates for 400+ LLMs agentops.ai Topics analytics price openai token price-tracker observability tokenization claude large-language-models llm Resources Readme License MIT license Activity Custom properties Stars 303 stars Watchers 7 watching Forks 16 forks Report repository Releases 14 0.1.10 Latest Jun 15, 2024 + 13 releases Packages 0 No packages published Contributors 6 * @areibman * @trisha * @AndreCNF * @oulianov * @scrpr * @henrikhargedahl Languages * Python 100.0% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.