Applied methods of analytical augmentation to build tools using large-language models.
$ npm install
$ make .env
Edit OPENAI_API_KEY and WOLFRAM_ALPHA_API_KEY.
To run the question-and-answer augmentation with a sample question:
$ make
The default question is the rather nonsensical What is the population of Geneseo, NY combined with the population of Rochester, NY, divided by string length of the answer to the question 'What is the capital of France?'?
The question-and-answer augmentation includes a number of exemplars expecting a user message:
Question: Context() ExistingArchivedFunctions() What is the population of Geneseo, NY combined with the population of Rochester, NY, divided by string length of the answer to the question 'What is the capital of France?'?
Which the assistant would respond with:
(async function solution(query, archiver) {
const [populationOfGeneseo, populationOfRochester, lengthOfCapital] =
await Promise.all([
query({
prompt: "What is the population of Geneseo, NY?",
topic: "Geneseo, NY",
target: "population",
type: "number",
}),
query({
prompt: "What is the population of Rochester, NY?",
topic: "Rochester, NY",
target: "population",
type: "number",
}),
query({
prompt: "What is the capital of France?",
topic: "France",
target: "capital",
type: "string",
}),
]);
const populationOfGeneseoPlusPopulationOfRochester =
populationOfGeneseo.answer + populationOfRochester.answer;
const answer =
populationOfGeneseoPlusPopulationOfRochester /
lengthOfCapital.answer.length;
return {
answer: answer,
solutions: [],
computed: true,
query: true,
};
});The query() function will use a number of external sources including Wikipedia, Wolfram Alpha, and Duck Duck Go combined with a nested call to the LLM to properly parse the query for further computation.
{
question: "What is the population of Geneseo, NY combined with the population of Rochester, NY, divided by string length of the answer to the question 'What is the capital of France?'?",
expectedAnswer: 43780.4,
answer: 43780.4,
en_answer: "The combined population of Geneseo, NY and Rochester, NY divided by the string length of the answer to the question 'What is the capital of France?' is 43892.6.",
correct: true
}
This means that this particular question made four total calls to the LLM. The first was to build the nested queries and the other three to parse the response from the search engines.
In essence the question-and-answer exemplars used in the few-shot instructions have taught the engine to use a JavaScript interpreter that has access to a specialized query function.
Let's ask another question:
What is the statistical variance for the list of numbers: [2, 4, 2, 5, 2, 6, 6]
This time the response is:
(async function solution(query, archiver) {
const numbers = [2, 4, 2, 5, 2, 6, 6];
function compute_variance(numbers) {
const mean = numbers.reduce((acc, curr) => acc + curr, 0) / numbers.length;
const variance = numbers.reduce((acc, curr) => acc + Math.pow(curr - mean, 2), 0) /
numbers.length;
return variance;
}
await archiver.add({
name: "compute_variance",
func: compute_variance,
argTypes: [{ numbers: "number[]" }],
returnType: "number",
description: `The function compute_variance takes an array of numbers as an input and computes the variance of those numbers. The function accomplishes this by first computing the mean of the input numbers, and then computing the variance of the input numbers by taking the sum of the squared differences between each number and the mean, and dividing by the number of input numbers.`,
isApplication: false,
});
const varianceOfNumbers = compute_variance(numbers);
return {
answer: varianceOfNumbers,
solutions: [],
computed: true,
query: false,
};
}
)Here the LLM has created a function and added it to the archiver. This function can be then be verfied and then used for further questions. If of course used this function on a given list of numbers to compute the variance:
2.979591836734694
The statistical variance for the list [2, 4, 2, 5, 2, 6, 6] is 2.979591836734694.
Another included augmentation assists with constructing web applications.
Given the following request:
make a new application called Draw50RandomCirclesWebApplication_v0_0_0
it should have a canvas element that is 320x480
the canvas element should have a black border and a drop shadow
the buttons should be the same size and have meaningful tasteful, pale colors, succinct and friendly text and emoji icons
it should have a button that draws 50 random circles when clicked
it should have a button that clears the canvas when clicked
use an instance of the Draw50RandomCirclesWebApplication to demonstrate to the user that it works
Becomes the augmented user prompt:
Context(<div id="context"></div>)
ExistingArchivedFunctions()
Instructions: make a new application called Draw50RandomCirclesWebApplication_v0_0_0
it should have a canvas element that is 320x480
the canvas element should have a black border and a drop shadow
the buttons should be the same size and have meaningful tasteful, pale colors, succinct and friendly text and emoji icons
it should have a button that draws 50 random circles when clicked
it should have a button that clears the canvas when clicked
use an instance of the Draw50RandomCirclesWebApplication to demonstrate to the user that it works -
You'll notice that there is HTML context provided. The browser-builder augmentation assumes that there is a browser document in scope during evaluation of the code returned by the LLM.
(async function solution(query, archiver, document) {
const Draw50RandomCirclesWebApplication_v0_0_0 = await archiver.build({
name: "Draw50RandomCirclesWebApplication_v0_0_0",
prompt:
"make a new application called Draw50RandomCirclesWebApplication_v0_0_0\n\nit should have a canvas element that is 320x480\nthe canvas element should have a black border and a drop shadow\nthe buttons should be the same size and have meaningful tasteful, pale colors, succinct and friendly text and emoji icons\nit should have a button that draws 50 random circles when clicked\nit should have a button that clears the canvas when clicked\n\nuse an instance of the Draw50RandomCirclesWebApplication to demonstrate to the user that it works",
argTypes: [
{ query: "any" },
{ archiver: "Archiver" },
{ document: "Document" },
],
returnType: "undefined",
description:
"A web application that draws 50 random circles on a canvas when the user clicks a button, and clears the canvas when another button is clicked.",
isApplication: true,
});
await Draw50RandomCirclesWebApplication_v0_0_0(query, archiver, document);
return {
answer: ["Draw50RandomCirclesWebApplication_v0_0_0"],
solutions: [],
computed: true,
query: false,
};
});However, instead of writing the JavaScript for our Draw50RandomCircles application directly this augmentation instead calls the archiver.build() function. Similar to the query function this will also make an additional call to an LLM. This build function stores the result in a database.
The prompt used by the build function:
Context((application) async function Draw50RandomCirclesWebApplication_v0_0_0(query: any, archiver: Archiver, document: Document): undefined [A web application that draws 50 random circles on a canvas when the user clicks a button, and clears the canvas when another button is clicked.])
ExistingArchivedFunctions()
Instructions: make a new application called Draw50RandomCirclesWebApplication_v0_0_0
it should have a canvas element that is 320x480
the canvas element should have a black border and a drop shadow
the buttons should be the same size and have meaningful tasteful, pale colors, succinct and friendly text and emoji icons
it should have a button that draws 50 random circles when clicked
it should have a button that clears the canvas when clicked
use an instance of the Draw50RandomCirclesWebApplication to demonstrate to the user that it works -
Results in:
(async function solution() {
async function Draw50RandomCirclesWebApplication_v0_0_0(
query,
archiver,
document
) {
const body = document.body;
const canvas = document.createElement("canvas");
canvas.width = 320;
canvas.height = 480;
canvas.style.border = "1px solid black";
canvas.style.boxShadow = "2px 2px 4px rgba(0, 0, 0, 0.4)";
const context = canvas.getContext("2d");
const buttonContainer = document.createElement("div");
const drawButton = document.createElement("button");
drawButton.innerText = "Draw 50 Random Circles 🟠";
drawButton.style.backgroundColor = "#F7D488";
drawButton.style.margin = "10px";
const clearButton = document.createElement("button");
clearButton.innerText = "Clear Canvas 🟥";
clearButton.style.backgroundColor = "#E37373";
clearButton.style.margin = "10px";
buttonContainer.appendChild(drawButton);
buttonContainer.appendChild(clearButton);
body.appendChild(canvas);
body.appendChild(buttonContainer);
drawButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
for (let i = 0; i < 50; ++i) {
const radius = Math.floor(Math.random() * 20) + 10;
const x =
Math.floor(Math.random() * (canvas.width - 2 * radius)) + radius;
const y =
Math.floor(Math.random() * (canvas.height - 2 * radius)) + radius;
const color = `rgb(${Math.floor(Math.random() * 256)}, ${Math.floor(
Math.random() * 256
)}, ${Math.floor(Math.random() * 256)})`;
context.beginPath();
context.arc(x, y, radius, 0, 2 * Math.PI);
context.fillStyle = color;
context.fill();
context.closePath();
}
};
clearButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
};
}
return {
answer: Draw50RandomCirclesWebApplication_v0_0_0,
en_answer: "Draw50RandomCirclesWebApplication_v0_0_0",
solutions: [],
computed: true,
};
});
Which is the function returned by archiver.build in the primary query.
The reason for the additional query is so that the dialog of messages isn't cluttered with JavaScript.
It turns out that we don't like the icons used, so we make an additional request:
Build a new version but choose different icons that are more visual and less abstract.
But we explictily reference the existing application:
Context(<div id="context"></div>)
ExistingArchivedFunctions()
Instructions: Reference: Draw50RandomCirclesWebApplication_v0_0_0 - Build a new version but choose different icons that are more visual and less abstract. -
Which results in:
(async function solution(query, archiver, document) {
const Draw50RandomCirclesWebApplication = await archiver.rebuild({
prompt:
"Reference: Draw50RandomCirclesWebApplication_v0_0_0 - Build a new version but choose different icons that are more visual and less abstract.",
name: "Draw50RandomCirclesWebApplication_v0_0_0",
});
return {
answer: ["Draw50RandomCirclesWebApplication_v0_0_0"],
solutions: [],
computed: true,
query: false,
};
});Here the archiver.rebuild function is used. This will make a nested call to the LLM but with the first version of the code:
Context(Rebuild: %%%async function Draw50RandomCirclesWebApplication_v0_0_0(query, archiver, document) {
const body = document.body;
const canvas = document.createElement("canvas");
canvas.width = 320;
canvas.height = 480;
canvas.style.border = "1px solid black";
canvas.style.boxShadow = "2px 2px 4px rgba(0, 0, 0, 0.4)";
const context = canvas.getContext("2d");
const buttonContainer = document.createElement("div");
const drawButton = document.createElement("button");
drawButton.innerText = "Draw 50 Random Circles 🟠";
drawButton.style.backgroundColor = "#F7D488";
drawButton.style.margin = "10px";
const clearButton = document.createElement("button");
clearButton.innerText = "Clear Canvas 🟥";
clearButton.style.backgroundColor = "#E37373";
clearButton.style.margin = "10px";
buttonContainer.appendChild(drawButton);
buttonContainer.appendChild(clearButton);
body.appendChild(canvas);
body.appendChild(buttonContainer);
drawButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
for (let i = 0; i < 50; ++i) {
const radius = Math.floor(Math.random() * 20) + 10;
const x = Math.floor(Math.random() * (canvas.width - 2 * radius)) + radius;
const y = Math.floor(Math.random() * (canvas.height - 2 * radius)) + radius;
const color = `rgb(${Math.floor(Math.random() * 256)}, ${Math.floor(Math.random() * 256)}, ${Math.floor(Math.random() * 256)})`;
context.beginPath();
context.arc(x, y, radius, 0, 2 * Math.PI);
context.fillStyle = color;
context.fill();
context.closePath();
}
};
clearButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
};
}%%%)
ExistingArchivedFunctions()
Instructions: Reference: Draw50RandomCirclesWebApplication_v0_0_0 - Build a new version but choose different icons that are more visual and less abstract. -
Which results in:
(async function solution() {
async function Draw50RandomCirclesWebApplication_v0_0_1(
query,
archiver,
document
) {
const body = document.body;
const canvas = document.createElement("canvas");
canvas.width = 320;
canvas.height = 480;
canvas.style.border = "1px solid black";
canvas.style.boxShadow = "2px 2px 4px rgba(0, 0, 0, 0.4)";
const context = canvas.getContext("2d");
const buttonContainer = document.createElement("div");
const drawButton = document.createElement("button");
drawButton.innerText = "🎨 Draw 50 Random Circles";
drawButton.style.backgroundColor = "#F7D488";
drawButton.style.margin = "10px";
const clearButton = document.createElement("button");
clearButton.innerText = "🧹 Clear Canvas";
clearButton.style.backgroundColor = "#E37373";
clearButton.style.margin = "10px";
buttonContainer.appendChild(drawButton);
buttonContainer.appendChild(clearButton);
body.appendChild(canvas);
body.appendChild(buttonContainer);
drawButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
for (let i = 0; i < 50; ++i) {
const radius = Math.floor(Math.random() * 20) + 10;
const x =
Math.floor(Math.random() * (canvas.width - 2 * radius)) + radius;
const y =
Math.floor(Math.random() * (canvas.height - 2 * radius)) + radius;
const color = `rgb(${Math.floor(Math.random() * 256)}, ${Math.floor(
Math.random() * 256
)}, ${Math.floor(Math.random() * 256)})`;
context.beginPath();
context.arc(x, y, radius, 0, 2 * Math.PI);
context.fillStyle = color;
context.fill();
context.closePath();
}
};
clearButton.onclick = () => {
context.clearRect(0, 0, canvas.width, canvas.height);
};
}
return {
answer: Draw50RandomCirclesWebApplication_v0_0_1,
en_answer: "Draw50RandomCirclesWebApplication_v0_0_1",
solutions: [],
computed: true,
};
});
The dialog of messages, including the initial exemplars used for system, user and assistant messages, consists of higher-level JavaScript that orchestrates additional LLM tools to write or rewrite the actual code. LLM tokens are a limited resource when it comes to interactivity and interactions can quickly reach the token limit. Another benefit of separating the build and rebuild functionality from a specific dialog is that an entirely new dialog can be started with the same code.