馃 Handle Anthropic RateLimitError - #45
Open
lcombaldieu wants to merge 2 commits into
Open
Conversation
Copy link
Copy Markdown
Sorry, something went wrong.
Handle Anthropic RateLimitError
Copy link
Copy Markdown
Member
|
There has already been a built-in mechanisms to retry after failure: https://the-pocket.github.io/PocketFlow/core_abstraction/node.html Does this address the Anthropic RateLimitError? |
Sorry, something went wrong.
Copy link
Copy Markdown
Author
|
I don't think so for some reason. Here is the error stack : ....
Fetched 64 files.
Identifying abstractions using LLM...
Identified 8 abstractions.
Analyzing relationships using LLM...
Analyzing relationships using LLM...
Analyzing relationships using LLM...
Analyzing relationships using LLM...
Analyzing relationships using LLM...
Traceback (most recent call last):
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/main.py", line 84, in <module>
main()
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/main.py", line 81, in main
tutorial_flow.run(shared)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 16, in run
return self._run(shared)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 50, in _run
def _run(self,shared): p=self.prep(shared); o=self._orch(shared); return self.post(shared,p,o)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 48, in _orch
while curr: curr.set_params(p); last_action=curr._run(shared); curr=copy.copy(self.get_next_node(curr,last_action))
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 13, in _run
def _run(self,shared): p=self.prep(shared); e=self._exec(p); return self.post(shared,p,e)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 33, in _exec
if self.cur_retry==self.max_retries-1: return self.exec_fallback(prep_res,e)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 28, in exec_fallback
def exec_fallback(self,prep_res,exc): raise exc
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/pocketflow/__init__.py", line 31, in _exec
try: return self.exec(prep_res)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/nodes.py", line 286, in exec
response = call_llm(prompt)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/utils/call_llm.py", line 90, in call_llm
response = client.messages.create(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_utils/_utils.py", line 275, in wrapper
return func(*args, **kwargs)
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/resources/messages/messages.py", line 953, in create
return self._post(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1336, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1013, in request
return self._request(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1102, in _request
return self._retry_request(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1151, in _retry_request
return self._request(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1102, in _request
return self._retry_request(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1151, in _retry_request
return self._request(
File "/Users/lcombaldieu/Workspace/Tutorial-Codebase-Knowledge/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1117, in _request
raise self._make_status_error_from_response(err.response) from None
anthropic.RateLimitError: Error code: 429 - {'type': 'error', 'error': {'type': 'rate_limit_error', 'message': 'This request would exceed the rate limit for your organization (c0a6fc51-0945-42c8-9012-bcb5bc0f8399) of 20,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits. You can see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase.'}}
Let me know if you think there is a more graceful way of handling this problem. My PR code fixed the issue |
Sorry, something went wrong.
Copy link
Copy Markdown
Author
|
By the way thank you for the awesome repo. Here is the doc I generated for our open source project. |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
|
Got it! I think what you can do is to increase the wait and retry time. |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
|
Your project is very cool! Would you mind sharing it on https://github.com/The-Pocket/Tutorial-Codebase-Knowledge/discussions? |
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Simple change to catch and handle Anthropic RateLimitError