How to send longer text inputs to ChatGPT API

+1 vote

We have a use case for ChatGPT in summarizing long pieces of text (speech-to-text conversations which can be over an hour).

However we find that the 4k token limit tends to lead to a truncation of the input text to say half or so due to the token limit.

Processing in parts does not seem to retain history of previous parts.

What options do we have for submitting a longer request which is over 4k tokens?

Mar 24, 2023 in ChatGPT by anonymous
• 990 points
3,065 views

2 answers to this question.

0 votes

here are a few options for submitting a longer request which is over 4k tokens.

Option 1: Use the file parameter.

As mentioned above, the file parameter can accept a file of any length. This means that you can upload your entire text file to the ChatGPT API and it will be processed in its entirety.

Option 2: Use the prompt parameter multiple times.

If you don't want to upload your entire text file to the ChatGPT API, you can use the prompt parameter multiple times to send the text in chunks. For example, you could send the first 2000 tokens in the first request, the next 2000 tokens in the second request, and so on.

Option 3: Use the stream parameter.

The stream parameter allows you to send the text to the ChatGPT API as a stream. This means that you can send the text one token at a time, which can be useful for very large text files.

Option 4: Use the batch parameter.

The batch parameter allows you to send multiple text files to the ChatGPT API at the same time. This can be useful if you have a large number of text files that you need to summarize.

Which option is best for you will depend on the specific needs of your use case. If you need to summarize a very large text file, then the stream parameter may be the best option. If you need to summarize a number of text files, then the batch parameter may be the best option.

I hope this helps! Let me know if you have any other questions.

Ready to master ChatGPT and unlock its full potential? Enroll in our comprehensive ChatGPT Course today!

answered May 11, 2023 by anonymous
0 votes

To send longer text inputs to the ChatGPT API, you can split your text into smaller chunks and send them as multiple requests. The API has a maximum token limit, which determines the length of text it can process in a single call. Here's a step-by-step guide on how to accomplish this:

  1. Determine the maximum token limit: The API response includes a usage field that provides information about the total tokens used and the maximum tokens allowed for your plan. Take note of the maximum tokens allowed value.

  2. Split your text: Break your longer text input into smaller chunks, ensuring that each chunk is within the maximum token limit. You can use the OpenAI Cookbook's tiktoken Python library (https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb) to count the number of tokens in your text and split it accordingly.

  3. Send requests for each chunk: Make individual API calls for each chunk of text. Start with the first chunk and continue until you have processed all the chunks.

  4. Preserve context between requests: To maintain a coherent conversation, you should include the context from previous requests in subsequent requests. The API response includes a context field, which you can update by appending the message from the previous request.

Here's an example using Python code to illustrate the process:

import openai

# Set up your OpenAI API credentials
openai.api_key = 'YOUR_API_KEY'

# Initialize the conversation with an empty string as context
context = ''

# Split your longer text into smaller chunks
text_chunks = ["This is the first chunk of text.", "This is the second chunk of text.", "And so on..."]

# Iterate through each chunk and send requests
for chunk in text_chunks:
    # Append the current chunk to the existing context
    input_text = context + chunk
    
    # Send the API request
    response = openai.Completion.create(
        engine='text-davinci-003',
        prompt=input_text,
        max_tokens=500,  # Adjust according to the maximum token limit for your plan
        temperature=0.7,
        n=1,
        stop=None,
        context=context
    )
    
    # Get the generated message from the response
    message = response.choices[0].text.strip()
    
    # Append the message to the context for the next iteration
    context += message

    # Process the generated message or store the results
    
    # Rest of your code...

Remember to replace 'YOUR_API_KEY' with your actual API key and adjust the max_tokens parameter according to your plan's token limit.

By splitting your longer text into smaller chunks and maintaining the context, you can effectively send longer inputs to the ChatGPT API.

Ready to master ChatGPT and unlock its full potential? Enroll in our comprehensive ChatGPT Course today!

answered May 22, 2023 by anonymous
• 1,180 points

Related Questions In ChatGPT

0 votes
1 answer

how to fix ChatGPT is at capacity right now on ChatGPT?

The message "ChatGPT is at capacity right ...READ MORE

answered Feb 9, 2023 in ChatGPT by Elton
836 views
0 votes
0 answers

How I can structure, format the ChatGPT response from api

I have integrated the chatgpt into my ...READ MORE

Mar 24, 2023 in ChatGPT by anonymous
• 990 points
876 views
–1 vote
1 answer

How can i make money from ChatGPT?

As an individual user, you cannot directly ...READ MORE

answered Feb 15, 2023 in ChatGPT by anonymous
695 views
0 votes
1 answer

Can I use ChatGPT to create chatbot on my website?

Yes, you can use ChatGPT to create ...READ MORE

answered Feb 15, 2023 in ChatGPT by anonymous
716 views
0 votes
1 answer

I keep getting a lot of errors on ChatGPT

Here are some of the common errors ...READ MORE

answered Feb 7, 2023 in ChatGPT by Elton
• 400 points
1,001 views
0 votes
1 answer

ChatGPT is not working

If ChatGPT is not working as expected, ...READ MORE

answered Feb 7, 2023 in ChatGPT by Elton
• 400 points
623 views
0 votes
1 answer

Should we edit answers that suggest OP ask ChatGPT?

Yes, it is generally not helpful to ...READ MORE

answered Mar 28, 2023 in ChatGPT by anonymous
• 300 points
316 views
0 votes
1 answer

How to supply relevant information to ChatGPT?

To supply relevant information to ChatGPT, you ...READ MORE

answered May 22, 2023 in ChatGPT by anonymous
• 1,180 points
270 views
0 votes
1 answer

ChatGPT showing blank white page after login

If you're seeing a blank white page, ...READ MORE

answered Apr 18, 2023 in ChatGPT by anonymous
• 1,180 points
3,630 views
0 votes
1 answer

NodeJS Amazon AWS S3 getObject how to send file in API response to download

Server Side const aws = require('aws-sdk'); router.get('/getfilefroms3', async (req, ...READ MORE

answered Mar 24, 2022 in Others by gaurav
• 23,260 points
9,220 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP