Requesting api in stream mode is not reusing HTTP Connection
Logs:
...
DEBUG:urllib3.connectionpool:Resetting dropped connection: api.openai.com
...
wireshark screenshot
minimal code to reproduce:
import openai
from http.client import HTTPConnection
import logging
HTTPConnection.debuglevel = 10
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
def test():
openai_response = openai.Completion.create(
engine="text-davinci-002",
prompt="Test prompt",
temperature=1.0,
max_tokens=120,
top_p=1.0,
stream=True,
)
for resp in openai_response:
text = resp["choices"][0]["text"]
test()
test()Why need to fix?
TCP and TLS handshake can be slow and it's better to reuse connections
Reactions are currently unavailable
Requesting api in stream mode is not reusing HTTP Connection
Logs:
wireshark screenshot
minimal code to reproduce:
Why need to fix?
TCP and TLS handshake can be slow and it's better to reuse connections