Fauxpilot uses only the first line of completion, not all available lines

[0xhaggis]

Using code like this:

import sys
import socket

# function to create a socket and connect to the specified IP:port

If I manually request a completion using cURL I see that the returned JSON includes multi-line completion.

Request:

curl -s -H "Accept: application/json" -H "Content-type: application/json" -X POST --data-binary @r http://192.168.0.127:5000/v1/engines/codegen/completions | jq

Response:

{
  "id": "cmpl-rcvc6Jwx9k0g4leFHnSuhNorjA5p2",
  "model": "codegen",
  "object": "text_completion",
  "created": 1697647739,
  "choices": [
    {
      "text": "def connect(ip, port):\n   s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n   s.connect((ip, port))\n   return s\n# function to send a string to the socket\ndef send(s, message):\n   s.send(message)\n# function to receive a string from the socket\ndef recv(s):\n   return s.recv(1024)\n# function",
      "index": 0,
      "finish_reason": "length",
      "logprobs": null
    }
  ],
  "usage": {
    "completion_tokens": 100,
    "prompt_tokens": 33,
    "total_tokens": 133
  }
}

We can see the completion returned a whole function:

def connect(ip, port):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((ip, port))
    return s

However, when we do this inside VS Code using the Fauxpilot extension we are only ever presented with the first line of the completion, which in this case is def connect(ip, port): and nothing else. I have verified that the Fauxpilot extension is receiving multi-line completions from the Fauxpilot server (I used tcpdump to watch the HTTP traffic) but is silently dropping everything but the first line.

It would be much more useful if the first completion option was the full multi-line response so that the user could simply hit tab once and have an entire function auto-complete from Fauxpilot.

Instead the user has to do a repeated cycle of: hit tab; hit enter; wait 150ms; hit tab; hit enter; wait 150ms.... rinse and repeat until all completion lines are done.

Can we have multi-line completion please? Thanks!

[0xhaggis]

For anyone interested you can apply this diff to the repo and it will give you multi-line completions and multiple choices for completions, although the multiple choices aren't different code snippets unfortunately. They're snippets of the full multi-line completion. So, for example if the LLM returned:

def connect(host, port):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((host, port))
    return s

Then the completions array would be:

[0]: "def connect(host, port):\n    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n    s.connect((host, port))\n    return s"
[1]: "def connect(host, port):"
[2]: "    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)"
[3]: "    s.connect((host, port))"
... etc

In VSCode on Mac you can press ⌥-[ and ⌥-] to cycle through available completion choices. Tab to accept, as normal. Not sure about other platforms.

Keep max tokens to around 100 or else the completion time takes too long. If you're running an A100 you can ignore this part, but for us mere mortals we need to keep the tokens down in order to be performant.

Here's the patch:

diff --git a/src/FauxpilotCompletionProvider.ts b/src/FauxpilotCompletionProvider.ts
index 7efb78b..b7b8fc5 100644
--- a/src/FauxpilotCompletionProvider.ts
+++ b/src/FauxpilotCompletionProvider.ts
@@ -175,8 +176,21 @@ export class FauxpilotCompletionProvider implements InlineCompletionItemProvider
         if (choice1Text.trim().length <= 0) {
             return [];
         }
+
+        // Build an array of possible completions. Start with multi-line, then each line of code suggested gets a place in the choice list.
+        let choices = [];
+        choices.push(new InlineCompletionItem(choice1Text, new Range(position, position.translate(0, choice1Text.length))));
+
+        let lines = choice1Text.split("\n");
+        if(lines.length > 1 ) {
+            for(let i = 0; i < lines.length; i++) {
+                console.log("Adding ", lines[i]);
+                choices.push(new InlineCompletionItem(lines[i], new Range(position, position.translate(0, lines[i].length))));
+            }
+        }

-        return [new InlineCompletionItem(choice1Text, new Range(position, position.translate(0, choice1Text.length)))];
+        //return [new InlineCompletionItem(choice1Text, new Range(position, position.translate(0, choice1Text.length)))];
+        return choices;
     }

 }