Tracking token usage 本筆記介紹了如何追蹤特定呼叫的tokens使用情況。目前只為OpenAI API實現。 讓我們首先看一個非常簡單的例子，追蹤單一LLM呼叫的tokens使用情況。 from langchain.callbacks import get_openai_callback from langchain_openai import OpenAI ''' llm = OpenAI(model_name="gpt-3.5-turbo-instruct", n=2, best_of=2) 在使用OpenAI的接口創建語言模型（LLM）實例時，參數`n=2`和`best_of=2`具有特定的含義： 1. `n=2`：這個參數指定了生成響應的數量。在這個例子中，`n=2`意味著每次請求模型時，它將生成兩個獨立的響應。這可以用於生成多個答案選項，從中選擇最合適的一個。 2. `best_of=2`：這個參數影響生成響應的選擇過程。當設置`best_of`參數時，系統會生成指定數量（在此例為2）的響應，然後根據內部評估標準選擇最佳的一個來返回。這個過程旨在提高響應的質量，因為系統有機會從多個生成的選項中選擇最佳的一個。然而，值得注意的是，當`n`和`best_of`的值相等時，系統將生成`n`個響應並從中選出最佳的一個，這基本上意味著每次請求都會嘗試提供最好的單一響應。 綜上所述，這種參數配置（`n=2, best_of=2`）使得每次請求時，模型都會生成兩個響應並從中選擇最好的一個來返回。這樣的設置有助於在不同生成的響應之間進行選擇，從而提高結果的質量，尤其是在需要最優解答的場景中。 ''' with get_openai_callback() as cb: result = llm.invoke("Tell me a joke") print(cb) Tokens Used: 37 Prompt Tokens: 4 Completion Tokens: 33 Successful Requests: 1 Total Cost (USD): $7.2e-05 在內容管理器內的任何內容都將被追蹤。這裡有一個用來追蹤連續多個呼叫的例子。 with get_openai_callback() as cb: result = llm.invoke("Tell me a joke") result2 = llm.invoke("Tell me a joke") print(cb.total_tokens) 72 如果使用了包含多個步驟的鏈或代理，它將追蹤所有這些步驟。 from langchain.agents import AgentType, initialize_agent, load_tools from langchain_openai import OpenAI llm = OpenAI(temperature=0) tools = load_tools(["serpapi", "llm-math"], llm=llm) agent = initialize_agent( tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True ) # 假設的代碼片段，模擬處理數學問題的過程 #https://serpapi.com/ # 定義二次方程的係數 with get_openai_callback() as cb: response = agent.run( "Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?" ) print(f"Total Tokens: {cb.total_tokens}") print(f"Prompt Tokens: {cb.prompt_tokens}") print(f"Completion Tokens: {cb.completion_tokens}") print(f"Total Cost (USD): ${cb.total_cost}") > Entering new AgentExecutor chain... I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.Action: SearchAction Input: "Olivia Wilde boyfriend"Observation: ["Olivia Wilde and Harry Styles took fans by surprise with their whirlwind romance, which began when they met on the set of Don't Worry Darling.", 'Olivia Wilde started dating Harry Styles after ending her years-long engagement to Jason Sudeikis — see their relationship timeline.', 'Olivia Wilde and Harry Styles were spotted early on in their relationship walking around London. (. Image ...', "Looks like Olivia Wilde and Jason Sudeikis are starting 2023 on good terms. Amid their highly publicized custody battle – and the actress' ...", 'The two started dating after Wilde split up with actor Jason Sudeikisin 2020. However, their relationship came to an end last November.', "Olivia Wilde and Harry Styles started dating during the filming of Don't Worry Darling. While the movie got a lot of backlash because of the ...", "Here's what we know so far about Harry Styles and Olivia Wilde's relationship.", 'Olivia and the Grammy winner kept their romance out of the spotlight as their relationship began just two months after her split from ex-fiancé ...', "Harry Styles and Olivia Wilde first met on the set of Don't Worry Darling and stepped out as a couple in January 2021. Relive all their biggest relationship ..."]Thought: Harry Styles is Olivia Wilde's boyfriend.Action: SearchAction Input: "Harry Styles age"Observation: 29 yearsThought: I need to calculate 29 raised to the 0.23 power.Action: CalculatorAction Input: 29^0.23Observation: Answer: 2.169459462491557Thought: I now know the final answer.Final Answer: Harry Styles is Olivia Wilde's boyfriend and his current age raised to the 0.23 power is 2.169459462491557.> Finished chain.Total Tokens: 2205Prompt Tokens: 2053Completion Tokens: 152Total Cost (USD): $0.0441 > 進入新的AgentExecutor鏈...我需要找出奧利維亞·懷爾德的男朋友是誰，然後計算他的年齡的0.23次方。行動：搜索行動輸入："Olivia Wilde boyfriend"觀察：["奧利維亞·懷爾德和哈利·斯泰爾斯讓粉絲們對他們的狂風戀情感到驚訝，這段戀情始於他們在《Don't Worry Darling》的拍攝現場相遇。", '奧利維亞·懷爾德在與傑森·蘇戴奇斯的多年訂婚關係結束後開始與哈利·斯泰爾斯交往——看看他們的戀情時間線。', '奧利維亞·懷爾德和哈利·斯泰爾斯在戀情初期就被發現在倫敦四處漫步。（圖片...）', "看來奧利維亞·懷爾德和傑森·蘇戴奇斯在2023年初關系良好。在他們引人注目的監護權戰爭中，這位女演員的...",'兩人在懷爾德於2020年與演員傑森·蘇戴奇斯分手後開始交往。然而，他們的關系在去年11月結束。', "奧利維亞·懷爾德和哈利·斯泰爾斯在拍攝《Don't Worry Darling》期間開始交往。雖然這部電影因為...而受到了很多抨擊。", "這是我們目前對哈利·斯泰爾斯和奧利維亞·懷爾德的關系所知道的。", '奧利維亞和這位格萊美獎得主在離開前未婚夫的兩個月後就開始戀情，並將他們的戀情保持在聚光燈之外。', "哈利·斯泰爾斯和奧利維亞·懷爾德首次在《Don't Worry Darling》的拍攝現場相遇，並在2021年1月以情侶身份亮相。回顧他們最大的戀情..."]思考：哈利·斯泰爾斯是奧利維亞·懷爾德的男朋友。行動：搜索行動輸入："Harry Styles age"觀察：29歲思考：我需要計算29的0.23次方。行動：計算器行動輸入：29^0.23觀察：答案：2.169459462491557思考：我現在知道最後的答案。最終答案：哈利·斯泰爾斯是奧利維亞·懷爾德的男朋友，他現在的年齡的0.23次方是2.169459462491557。> 完成鏈。總代幣：2205提示代幣：2053完成代幣：152總成本（美元）：$0.0441