Guide to Using the GPT-3.5 Turbo API: Parameters and Code Examples (2023)

Introduction

Today (2023.03.02), OpenAI released the latest GPT-3.5 Turbo API, currently priced at $0.002/1k tokens¹.

warning
The information in this article is time-sensitive; please verify accordingly.

Usage Guide

The official documentation is here², which is obviously more detailed than what I’ve written 233. Unless something unexpected happens, I still recommend checking the official docs.

After writing this, I realized it might actually be more detailed than the official docs! 233

Registration

warning
You may need a free internet environment for this. Also, if you are using proxy software, you must set `HTTP_PROXY` and `HTTPS_PROXY` in your command-line environment when running the program below, otherwise you will encounter access errors. On Windows, you can use the `set` command, `set HTTP_PROXY=http://127.0.0.1:xxxx`; on Linux, you can use the `export` command, `export HTTP_PROXY=http://127.0.0.1:xxxx`, where `xxxx` is the proxy software port number.

First, register for a developer account on OpenAI Platform, then generate an API Key on the API Keys page.

Currently, OpenAI provides $18 in free credits for one month, which should be more than enough for testing.

Demo Example

Let’s get started! Before beginning, you need to install the openai library; the current latest version is v0.27.0.

1
pip install openai

If you have installed it before, you might need to upgrade it.

1
pip install openai --upgrade

Below is a demo provided by the official documentation; simply replace the API key text with your own, and it will run directly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # replace with your API key
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}]
)

print(completion.choices[0].message)

This way, we have successfully greeted GPT-3.5 Turbo!

Advanced Usage # 1

Let’s look at a longer example first.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # replace with your API key
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the world series in 2020?"},
        {
            "role": "assistant",
            "content": "The Los Angeles Dodgers won the World Series in 2020.",
        },
        {"role": "user", "content": "Where was it played?"},
    ],
)

print(completion.choices[0].message)

First, you need to use the ChatCompletion from the openai library, then call the create method to create a ChatCompletion object, which contains our request information.

Let’s observe the request format:

model: No need to explain this; it is the model name. Currently, we are testing the gpt-3.5-turbo model, and only two models are supported: gpt-3.5-turbo and gpt-3.5-turbo-0301. Models with dates in their names will not be updated, but for today, the two are identical.
messages: This is a list where each element is a dictionary. The role in the dictionary represents the message. Currently, three types are supported: user, system, and assistant. content represents the content of the message.
- system: System message, used to set the behavior of ChatGPT.
- user: User message, used to interact with ChatGPT.
- assistant: Assistant message, used to help store ChatGPT’s previous responses.

Let’s look at the complete response.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The 2020 World Series was played at Globe Life Field in Arlington, Texas.",
        "role": "assistant"
      }
    }
  ],
  "created": 1677759714,
  "id": "chatcmpl-6pcEUG6zKP0Ld33OP1dZGbg3GwWhC",
  "model": "gpt-3.5-turbo-0301",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 19,
    "prompt_tokens": 56,
    "total_tokens": 75
  }
}

This is actually an example of multi-turn conversation. If you want to dynamically conduct multi-turn conversations, you must record and pass all previous responses each time.

First, we set the system message, whose content is You are a helpful assistant.. Then we set the user message, whose content is Who won the world series in 2020?; this message serves as the information from the previous turn. Next, we set the assistant message, whose content is The Los Angeles Dodgers won the World Series in 2020., meaning ChatGPT’s response in the previous turn. Finally, we set the user message, whose content is Where was it played?, representing the question for this turn, i.e., the question we currently want GPT to answer.

As you can see, the received response content is The 2020 World Series was played at Globe Life Field in Arlington, Texas.. We only asked about the location, but ChatGPT already knew from the previous responses that this turn was about the 2020 World Series and answered accordingly.

At the same time, we can also see the usage field, which indicates how many tokens were used in this request: prompt_tokens is the number of input tokens, completion_tokens is the number of tokens in the ChatGPT response, and total_tokens is the total number of tokens used. This round consumed 75 tokens.

Advanced Usage # 2

Let’s look at a more complex example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # replace with your API key
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": 'I want you to act as an Chinese translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in Chinese. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level Chinese words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. My first sentence is "Non terrae plus ultra"',
        },
    ],
    temperature=0.9,  # 0.0 to 2.0 (default 1.0)
    top_p=1,  # 0.0 to 1.0 (default 1.0) (not used if temperature is set)
    n=5,  # number (default 1) How many chat completion choices to generate for each input message.
    stream=False,  # boolean (default False)
    stop=None,  # string or array (default None)
    max_tokens=10,  # inf (default 4096-prompt_token)
    presence_penalty=2.0,  # -2.0 to 2.0 (default 0)
    frequency_penalty=0,  # -2.0 to 2.0 (default 0)
    # logit_bias=
    # user=
)

print(completion)

for choice in completion.choices:
    print(choice.message.content)

The prompt here is adapted from ³; the prompt itself is unrelated to the parameters I’m explaining.

This involves more parameters:

temperature: 0.0 to 2.0 (default 1.0) Temperature. Higher values make the output more random; lower values make it more deterministic (or regular).
top_p: 0.0 to 1.0 (default 1.0) An alternative to temperature, also known as nucleus sampling. It is recommended not to use temperature and top_p simultaneously. top_p indicates that the model only considers the top top_p tokens by probability. For example, top_p=0.1 means the model only considers the top 10% of tokens by probability.
n: number (default 1) The number of responses to generate.
stream: boolean (default False) Whether to use streaming mode. If set to True, partial message chunks will be sent incrementally, just like in ChatGPT. What does that mean? It means a few words are sent to you one by one, allowing you to dynamically update the text as you wait for the full response in ChatGPT.
stop: string or array (default None) Tokens used to stop generation. It can be a single string or a list of strings. If it’s a list, generation stops as soon as any one of the tokens appears, with a maximum of 4 tokens allowed.
max_tokens: inf (default 4096 - prompt_token) The maximum number of tokens to generate.
frequency_penalty and presence_penalty: -2.0 to 2.0 (default 0) Used to penalize repeated tokens. More details about these parameters are available in ⁴. One appears to handle frequency, while the other handles presence (as an integer). The higher the values of these two parameters, the less likely the generated text will repeat.

The formula is as follows:

1
mu[j] -> mu[j] - c[j] * alpha_frequency - float(c[j] > 0) * alpha_presence

logit_bias: dict (default None) Used to adjust token probabilities; accepts JSON. Values range from -100 to 100. -100 effectively disables the word, while 100 forces its use if relevant.
user: dict (default None) Used to set user information. For specifics, refer to ⁵, mainly to prevent abuse.

The output of this code is as follows (since Chinese characters are escaped in JSON, I’ve replaced them with placeholders here).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "\"天涯海角\"",
        "role": "assistant"
      }
    },
    {
      "finish_reason": "stop",
      "index": 1,
      "message": {
        "content": "\n\n无穷尽之地",
        "role": "assistant"
      }
    },
    {
      "finish_reason": null,
      "index": 2,
      "message": {
        "content": "\n\n\"无出其右\"",
        "role": "assistant"
      }
    },
    {
      "finish_reason": "stop",
      "index": 3,
      "message": {
        "content": "\n\n\"无地不可至\"",
        "role": "assistant"
      }
    },
    {
      "finish_reason": "length",
      "index": 4,
      "message": {
        "content": "\n\n无地可往，更远",
        "role": "assistant"
      }
    }
  ],
  "created": 1677760749,
  "id": "chatcmpl-6pcVBkXoD9xr2CSioR1Gz3ubEOdpg",
  "model": "gpt-3.5-turbo-0301",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 45,
    "prompt_tokens": 124,
    "total_tokens": 169
  }
}

Advanced Usage # 3

Here’s an example regarding the steam parameter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": 'I want you to act as an Chinese translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in Chinese. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level Chinese words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. My first sentence is "Non terrae plus ultra"',
        },
    ],
    temperature=1,  # 0.0 to 2.0 (default 1.0)
    top_p=1,  # 0.0 to 1.0 (default 1.0) (not used if temperature is set)
    n=1,  # number (default 1) How many chat completion choices to generate for each input message.
    stream=True,  # boolean (default False)
    stop=None,  # string or array (default None)
    # max_tokens=100,  # inf (default 4096-prompt_token)
    presence_penalty=2.0,  # -2.0 to 2.0 (default 0)
    frequency_penalty=0,  # -2.0 to 2.0 (default 0)
    # logit_bias=
    # user=
)

for completion_ in completion:
    # print(completion_)
    for choice in completion_.choices:
        print(choice.delta.content if "content" in choice.delta else "")

When streaming mode is enabled, the response will be a stream of data rather than a single object containing all data, and this returned object is iterable. Below is an example of a returned item. Note that delta may not always contain content, so a check is necessary.

In other words, ChatGPT actually already has the complete result before outputting it; it just spits out words one by one to drag out the time on the frontend?!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
  "choices": [
    {
      "delta": {
        "content": "\uff1f"
      },
      "finish_reason": null,
      "index": 0
    }
  ],
  "created": 1677763452,
  "id": "chatcmpl-6pdCm5jwsB1e3YyEDZ1MQXbpHzWvn",
  "model": "gpt-3.5-turbo-0301",
  "object": "chat.completion.chunk"
}

Advanced Usage # 4

Here’s an example regarding the stop parameter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": 'I want you to act as an Chinese translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in Chinese. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level Chinese words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. My first sentence is "Non terrae plus ultra"',
        },
    ],
    temperature=1,  # 0.0 to 2.0 (default 1.0)
    top_p=1,  # 0.0 to 1.0 (default 1.0) (not used if temperature is set)
    n=1,  # number (default 1) How many chat completion choices to generate for each input message.
    stream=False,  # boolean (default False)
    stop="无",  # string or array (default None)
    # max_tokens=100,  # inf (default 4096-prompt_token)
    presence_penalty=0,  # -2.0 to 2.0 (default 0)
    frequency_penalty=0,  # -2.0 to 2.0 (default 0)
    # logit_bias=
    # user=
)

for choice in completion.choices:
    print(choice.message.content)

Here, we still have it play the role of a translator to translate a line from Apex Legends spoken by Octane. We use “无” (none) as the stop condition; when the output encounters “无”, generation stops and the result is returned. The output is as follows:

1
您好，您所给出的第一句话“Non terrae plus ultra”是拉丁语，意为“没有比这更远的土地”，翻译成中文可写作“

As you can see, the output is directly cut off, and the interrupted word is automatically converted into a token. However, I haven’t yet thought of any practical application for this –

Error

I encountered this error several times while using it. It wasn’t due to issues with my request format; it was likely because too many people were making requests simultaneously. The error message looks like this:

1
2
3
4
5
6
7
8
9
openai.error.APIError: The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 7d8d2f6b67d92ff7850ef3e17d742827 in your email.) {
  "error": {
    "message": "The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 7d8d2f6b67d92ff7850ef3e17d742827 in your email.)",
    "type": "server_error",
    "param": null,
    "code": null
  }
}
 500 {'error': {'message': 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 7d8d2f6b67d92ff7850ef3e17d742827 in your email.)', 'type': 'server_error', 'param': None, 'code': None}} {'Date': 'Thu, 02 Mar 2023 12:37:21 GMT', 'Content-Type': 'application/json', 'Content-Length': '366', 'Connection': 'keep-alive', 'Access-Control-Allow-Origin': '*', 'Openai-Model': 'gpt-3.5-turbo-0301', 'Openai-Organization': 'user-8qnumkqgd3l02hvzq5rqz0y1', 'Openai-Processing-Ms': '750', 'Openai-Version': '2020-10-01', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains', 'X-Request-Id': '7d8d2f6b67d92ff7850ef3e17d742827'}

Conclusion

I can only say this pricing is incredibly cheap. It feels like many companies won’t even bother trying to replicate it themselves. Instead, they’ll just use the library. It offers good performance, no need to worry about costs, electricity, compute power, or other factors, and the price is low. I feel that even small companies with their own models might find that compute and electricity costs alone are significantly higher than the API. After all, this also comes down to utilization rates.

Additionally, I feel this has also changed the translation market to some extent. Taking Tencent Cloud’s translation API as an example, if you calculate the price roughly, it’s about 3 times that of the GPT-3.5 Turbo API. However, including input tokens, it’s more like 1.5 times, and it comes with other features, including rewriting and polishing.

Through experience, you can also find that if you input long texts every time, tokens are consumed quite quickly, as both input and output are billed. At the same time, if you want to conduct session-level conversations, your token consumption will grow rapidly: each turn multiplies by 2, and when accumulated, it becomes quadratic. So, the cost of long conversations is actually quite significant.

However, unfortunately, OpenAI currently only supports virtual credit card payments. Domestic users who want to pay out of pocket may need to find their own solutions.

Do you remember that last September I wrote a blog post about my thoughts on Stable Diffusion⁶? Now, looking at the SD models again, they seem to be a century behind… LoRA, ControlNet… If SD only impacted the art and design fields, then the potential of ChatGPT’s large models is truly vast and will affect many industries, because the diversity of outputs is incredibly rich. For example, someone might use it to generate code to drive machines, and so on⁷… Application scenarios depend entirely on imagination. However, there are currently scientific issues. If a more powerful knowledge base could be established to provide theoretical backing during output, the application scenarios of this model would become even broader, such as in healthcare, finance, and other fields with higher demands for evidence and decision-making.

Guide to Using the GPT-3.5 Turbo API: Parameters and Code Examples (2023)

Introduction

Usage Guide

Registration

Demo Example

Advanced Usage # 1

Advanced Usage # 2

Advanced Usage # 3

Advanced Usage # 4

Error

Conclusion

Author

Published

Updated

License

Introduction#

Usage Guide#

Registration#

Demo Example#

Advanced Usage # 1#

Advanced Usage # 2#

Advanced Usage # 3#

Advanced Usage # 4#

Error#

Conclusion#

Introduction

Usage Guide

Registration

Demo Example

Advanced Usage # 1

Advanced Usage # 2

Advanced Usage # 3

Advanced Usage # 4

Error

Conclusion