Merge pull request openai#85 from openai/ted/various-updates-and-addi…

…tions

Ted/various updates and additions

code_editing_examples.md

@@ -0,0 +1,63 @@
+# Code editing example
+
+OpenAI's [edits](https://openai.com/blog/gpt-3-edit-insert/) endpoint is particularly useful for editing code.
+
+Unlike completions, edits takes two inputs: the text to edit and an instruction.
+
+For example, if you wanted to edit a Python function, you could supply the text of the function and an instruction like "add a docstring".
+
+Example text input to `code-davinci-edit-001`:
+
+```python
+def tribonacci(n):
+    if n == 0:
+        return 0
+    elif n == 1:
+        return 1
+    elif n == 2:
+        return 1
+    elif n == 3:
+        return 2
+    else:
+        return tribonacci(n-1) + tribonacci(n-2) + tribonacci(n-3)
+```
+
+Example instruction inputs:
+
+```text
+add a docstring
+```
+
+```text
+Add typing, using Python 3.9 conventions
+```
+
+```text
+improved the runtime
+```
+
+```text
+Add a test.
+```
+
+```text
+Translate to JavaScript (or Rust or Lisp or any language you like)
+```
+
+Example output after improving the runtime and translating to JavaScript:
+
+```JavaScript
+function tribonacci(n) {
+  let a = 0;
+  let b = 1;
+  let c = 1;
+  for (let i = 0; i < n; i++) {
+    [a, b, c] = [b, c, a + b + c];
+  }
+  return a;
+}
+```
+
+As you can see, `code-davinci-edit-001` was able to successfully reduce the function's runtime from exponential down to linear, as well as convert from Python to JavaScript.
+
+Experiment with code editing using `code-davinci-edit-001` in the [OpenAI Playground](https://beta.openai.com/playground?mode=edit&model=code-davinci-edit-001).

code_explanation_examples.md

@@ -0,0 +1,41 @@
+# Code explanation examples
+
+GPT's understanding of code can be applied to many use cases, e.g.:
+
+* Generating in-code documentation (e.g., Python docstrings, git commit messages)
+* Generating out-of-code documentation (e.g., man pages)
+* An interactive code exploration tool
+* Communicating program results back to users via a natural language interface
+
+For example, if you wanted to understand a SQL query, you could give `code-davinci-002` the following example prompt:
+
+````text
+A SQL query:
+```
+SELECT c.customer_id
+FROM Customers c
+JOIN Streaming s
+ON c.customer_id = s.customer_id
+WHERE c.signup_date BETWEEN '2020-03-01' AND '2020-03-31'
+AND s.watch_date BETWEEN c.signup_date AND DATE_ADD(c.signup_date, INTERVAL 30 DAY)
+GROUP BY c.customer_id
+HAVING SUM(s.watch_minutes) > 50 * 60
+```
+
+Questions:
+1. What does the SQL query do?
+2. Why might someone be interested in this time period?
+3. Why might a company be interested in this SQL query?
+
+Answers:
+````
+
+[Output]((https://beta.openai.com/playground/p/Sv1VQKbJV1TZKmiTK9r6nlj3)):
+
+```text
+1. The SQL query finds all customers who signed up in March 2020 and watched more than 50 hours of content in the first 30 days after signing up.
+2. The time period is interesting because it is the first month of the COVID-19 pandemic.
+3. A company might be interested in this SQL query because it can help them understand how the pandemic has affected their business.
+```
+
+Note that `code-davinci-002` is not trained to follow instructions and therefore usually needs examples or other structure to help steer its output, as well as stop sequences to stop generating. For easier prompting, try `text-davinci-003`.

code_writing_examples.md

@@ -0,0 +1,31 @@
+# Code writing examples
+
+GPT-3 is able to write code as well as text.
+
+Here's an example of `code-davinci-002` writing a SQL query:
+
+````text
+SQL tables (and columns):
+* Customers(customer_id, signup_date)
+* Streaming(customer_id, video_id, watch_date, watch_minutes)
+
+A well-written SQL query that lists customers who signed up during March 2020 and watched more than 50 hours of video in their first 30 days:
+```
+````
+
+[Output](https://beta.openai.com/playground/p/r2mw99cANoa0TJHok725CeaC):
+
+```sql
+SELECT c.customer_id
+FROM Customers c
+JOIN Streaming s
+ON c.customer_id = s.customer_id
+WHERE c.signup_date BETWEEN '2020-03-01' AND '2020-03-31'
+AND s.watch_date BETWEEN c.signup_date AND DATE_ADD(c.signup_date, INTERVAL 30 DAY)
+GROUP BY c.customer_id
+HAVING SUM(s.watch_minutes) > 50 * 60
+```
+
+Helpfully, `code-davinci-002` is able to make inferences from variable names; for example, it infers that `watch_minutes` has units of minutes and therefore needs to be converted by a factor of 60 before being compared with 50 hours.
+
+For easier prompting, you can also try `text-davinci-003`.

examples/How_to_handle_rate_limits.ipynb

@@ -1,31 +1,37 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# How to handle rate limits\n",
     "\n",
     "When you call the OpenAI API repeatedly, you may encounter error messages that say `429: 'Too Many Requests'` or `RateLimitError`. These error messages come from exceeding the API's rate limits.\n",
     "\n",
+    "This guide shares tips for avoiding and handling rate limit errors.\n",
+    "\n",
+    "To see an example script for throttling parallel requests to avoid rate limit errors, see [api_request_parallel_processor.py](api_request_parallel_processor.py).\n",
+    "\n",
+    "## Why rate limits exist\n",
+    "\n",
     "Rate limits are a common practice for APIs, and they're put in place for a few different reasons.\n",
     "\n",
     "- First, they help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.\n",
     "- Second, rate limits help ensure that everyone has fair access to the API. If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that everyone has an opportunity to use the API without experiencing slowdowns.\n",
     "- Lastly, rate limits can help OpenAI manage the aggregate load on its infrastructure. If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.\n",
     "\n",
-    "Although hitting rate limits can be frustrating, rate limits exist to protect the reliable operation of the API for its users.\n",
-    "\n",
-    "In this guide, we'll share some tips for avoiding and handling rate limit errors."
+    "Although hitting rate limits can be frustrating, rate limits exist to protect the reliable operation of the API for its users."
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Default rate limits\n",
     "\n",
-    "As of Sep 2022, the default rate limits are:\n",
+    "As of Jan 2023, the default rate limits are:\n",
     "\n",
     "<table>\n",
     "<thead>\n",
@@ -56,7 +62,7 @@
     "    <td>\n",
     "        <ul>\n",
     "            <li>60 requests / minute</li>\n",
-    "            <li>250,000 davinci tokens / minute (and proportionally more for smaller models)</li>\n",
+    "            <li>250,000 davinci tokens / minute (and proportionally more for cheaper models)</li>\n",
     "        </ul>\n",
     "    </td>\n",
     "    <td>\n",
@@ -71,7 +77,7 @@
     "    <td>\n",
     "        <ul>\n",
     "            <li>3,000 requests / minute</li>\n",
-    "            <li>250,000 davinci tokens / minute (and proportionally more for smaller models)</li>\n",
+    "            <li>250,000 davinci tokens / minute (and proportionally more for cheaper models)</li>\n",
     "        </ul>\n",
     "    </td>\n",
     "    <td>\n",
@@ -88,16 +94,17 @@
     "\n",
     "### Other rate limit resources\n",
     "\n",
-    "Read more about OpenAI's rate limits in the [OpenAI Help Center](https://help.openai.com/en/):\n",
+    "Read more about OpenAI's rate limits in these other resources:\n",
     "\n",
-    "- [Is API usage subject to any rate limits?](https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits)\n",
-    "- [How can I solve 429: 'Too Many Requests' errors?](https://help.openai.com/en/articles/5955604-how-can-i-solve-429-too-many-requests-errors)\n",
+    "- [Guide: Rate limits](https://beta.openai.com/docs/guides/rate-limits/overview)\n",
+    "- [Help Center: Is API usage subject to any rate limits?](https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits)\n",
+    "- [Help Center: How can I solve 429: 'Too Many Requests' errors?](https://help.openai.com/en/articles/5955604-how-can-i-solve-429-too-many-requests-errors)\n",
     "\n",
     "### Requesting a rate limit increase\n",
     "\n",
     "If you'd like your organization's rate limit increased, please fill out the following form:\n",
     "\n",
-        "- [OpenAI Rate Limit Increase Request form](https://forms.gle/56ZrwXXoxAN1yt6i9)\n"
+    "- [OpenAI Rate Limit Increase Request form](https://forms.gle/56ZrwXXoxAN1yt6i9)\n"
    ]
   },
   {
@@ -379,6 +386,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -392,7 +400,7 @@
     "\n",
     "If you are constantly hitting the rate limit, then backing off, then hitting the rate limit again, then backing off again, it's possible that a good fraction of your request budget will be 'wasted' on requests that need to be retried. This limits your processing throughput, given a fixed rate limit.\n",
     "\n",
-    "Here, one potential solution is to calculate your rate limit and add a delay equal to its reciprocal (e.g., if your rate limit 20 requests per minute, add a delay of 3 seconds to each request). This can help you operate near the rate limit ceiling without hitting it and incurring wasted requests.\n",
+    "Here, one potential solution is to calculate your rate limit and add a delay equal to its reciprocal (e.g., if your rate limit 20 requests per minute, add a delay of 3–6 seconds to each request). This can help you operate near the rate limit ceiling without hitting it and incurring wasted requests.\n",
     "\n",
     "#### Example of adding delay to a request"
    ]
@@ -570,6 +578,25 @@
     "for story in stories:\n",
     "    print(story)\n"
    ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example parallel processing script\n",
+    "\n",
+    "We've written an example script for parallel processing large quantities of API requests: [api_request_parallel_processor.py](api_request_parallel_processor.py).\n",
+    "\n",
+    "The script combines some handy features:\n",
+    "- Streams requests from file, to avoid running out of memory for giant jobs\n",
+    "- Makes requests concurrently, to maximize throughput\n",
+    "- Throttles both request and token usage, to stay under rate limits\n",
+    "- Retries failed requests, to avoid missing data\n",
+    "- Logs errors, to diagnose problems with requests\n",
+    "\n",
+    "Feel free to use it as is or modify it to suit your needs."
+   ]
   }
  ],
  "metadata": {
@@ -588,7 +615,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.9.9 (main, Dec  7 2021, 18:04:56) \n[Clang 13.0.0 (clang-1300.0.29.3)]"
   },
   "orig_nbformat": 4,
   "vscode": {

examples/How_to_stream_completions.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -14,8 +15,14 @@
     "\n",
     "To stream completions, set `stream=True` when calling the Completions endpoint. This will return an object that streams back text as [data-only server-sent events](https://app.mode.com/openai/reports/4fce5ba22b5b/runs/f518a0be4495).\n",
     "\n",
+    "## Downsides\n",
+    "\n",
     "Note that using `stream=True` in a production application makes it more difficult to moderate the content of the completions, which has implications for [approved usage](https://beta.openai.com/docs/usage-guidelines).\n",
     "\n",
+    "Another small drawback of streaming responses is that the response no longer includes the `usage` field to tell you how many tokens were consumed. After receiving and combining all of the responses, you can calculate this yourself using [`tiktoken`](How_to_count_tokens_with_tiktoken.ipynb).\n",
+    "\n",
+    "## Example code\n",
+    "\n",
     "Below is a Python code example of how to receive streaming completions."
    ]
   },
@@ -31,10 +38,11 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## A typical completion request\n",
+    "### A typical completion request\n",
     "\n",
     "With a typical Completions API call, the text is first computed and then returned all at once."
    ]
@@ -80,10 +88,11 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## A streaming completion request\n",
+    "### A streaming completion request\n",
     "\n",
     "With a streaming Completions API call, the text is sent back via a series of events. In Python, you can iterate over these events with a `for` loop."
    ]
@@ -328,10 +337,11 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Time comparison\n",
+    "### Time comparison\n",
     "\n",
     "In the example above, both requests took about 7 seconds to fully complete.\n",
     "\n",
@@ -355,7 +365,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.9"
+   "version": "3.9.9 (main, Dec  7 2021, 18:04:56) \n[Clang 13.0.0 (clang-1300.0.29.3)]"
   },
   "orig_nbformat": 4,
   "vscode": {