Skip to content

Commit

Permalink
Fix prompts.
Browse files Browse the repository at this point in the history
  • Loading branch information
haotian-liu committed May 1, 2023
1 parent ad5fda9 commit 54bcf81
Show file tree
Hide file tree
Showing 5 changed files with 88 additions and 20 deletions.
12 changes: 5 additions & 7 deletions playground/data/prompts/conversation/000_caps.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
A restaurant has modern wooden tables and chairs.
A long restaurant table with rattan rounded back chairs.
a long table with a plant on top of it surrounded with wooden chairs
A long table with a flower arrangement in the middle for meetings
A table is adorned with wooden chairs with blue accents.

Include 5 rounds of QA. Include a question asking for a detailed description of the photo. Rephrase the request.
There is a movie theater that displays the show times above the doors.
A red fire hydrant is deep in the snow.
The fire hydrant is in the snow near a recently plowed sidewalk.
This city has had a very hard winter with snow.
A hotel for dogs in the snow in winter.
34 changes: 29 additions & 5 deletions playground/data/prompts/conversation/000_conv.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
{"question": "What is this?", "answer": "It is a restaurant with tables and chairs."}
{"question": "What type of chairs surround the long table?", "answer": "Rattan rounded back chairs."}
{"question": "What is under the long table?", "answer": "Hmm, I do not see anything that is under the long wooden table, but there is a plant on top of it."}
{"question": "What kind of atmosphere do you think the restaurant is trying to create?", "answer": "Looking at the photo, the restaurant is likely trying to create a warm and inviting atmosphere with the modern wooden tables and chairs. The use of wood can suggest a natural and rustic vibe, while the modern design of the furniture adds a touch of sophistication. The rattan rounded back chairs could also indicate a tropical or beachy feel. The presence of plants and flower arrangements further emphasizes the natural element and adds a cozy and welcoming feel to the space. Overall, the restaurant seems to be creating a comfortable and relaxing environment that would be enjoyable to dine in."}
{"question": "Can you provide a short paragraph describing the scene?", "answer": "The restaurant features modern wooden tables and chairs, creating a warm and stylish atmosphere. One long table is set up in the space, surrounded by rattan rounded back chairs, and wooden chairs with blue accents. A beautiful plant is placed on top of the table, adding a touch of natural elegance, while a separate long table showcases an intricate flower arrangement in the middle, which seems to be perfect for meetings. The careful combination of elements in the scene results in a well-appointed and inviting dining area."}
Question:
What color is the fire hydrant in the image?
===
Answer:
The fire hydrant in the image is red.
===
Question:
Is the fire hydrant fully covered by snow or partially visible?
===
Answer:
The fire hydrant is partially visible, with a significant portion of it covered by the deep snow.
===
Question:
What is the condition of the sidewalk near the fire hydrant?
===
Answer:
The sidewalk near the fire hydrant has been recently plowed, so it is relatively clear of snow compared to the surrounding area.
===
Question:
What does the image suggest about the winter season in the city?
===
Answer:
The image suggests that the winter season in the city has been quite harsh and challenging. The significant snow accumulation seen in the image indicates that the city has experienced heavy snowfall, which can lead to various difficulties and disruptions for its residents.

For instance, the deep snow covering the fire hydrant could potentially hinder its accessibility in case of emergencies, which poses safety concerns. The recently plowed sidewalk implies that the city's maintenance crews have been actively working to keep the walkways clear and safe for pedestrians, but the sheer amount of snowfall might make it difficult to keep up with the continuous snow removal efforts.

Furthermore, such extreme winter conditions can impact transportation, leading to traffic issues, delays in public transit, and increased risks of accidents due to slippery roads. It can also cause problems for businesses and schools, as people might face challenges commuting to work or attending classes. Additionally, the heavy snow can put extra strain on infrastructure, such as roofs and power lines, increasing the likelihood of structural damage or power outages.

In conclusion, the image of the red fire hydrant deep in the snow and the recently plowed sidewalk suggest that the city has faced a particularly severe winter season, with substantial snowfall that has likely caused various challenges and disruptions for its residents and infrastructure.
5 changes: 5 additions & 0 deletions playground/data/prompts/conversation/001_caps.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
A man is skiing in the open snow covered hills
A skier is making his way into the snow.
A skier on the foothills of a huge mountain range.
A skier looks at mountains while standing near a trail sign.
a single person skiing by an area with a lot of bushes
37 changes: 37 additions & 0 deletions playground/data/prompts/conversation/001_conv.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
Question:
What is the skier doing?
===
Answer:
The skier is skiing in the snow-covered hills, making his way through the snow and enjoying the beautiful mountain scenery. They are also standing near a trail sign, which indicates their location on the mountain.
===
Question:
Are there any other people or skiers visible in the image?
===
Answer:
No, there is only a single person skiing in the image, and they are surrounded by an area with a lot of bushes.
===
Question:
What kind of skiing is the person doing? Cross-country or downhill?
===
Answer:
Based on what I can see in the photo, it is not possible to definitively determine whether the skier is doing cross-country or downhill skiing. However, given the presence of a trail sign and the fact that they are skiing in the foothills, it is more likely that the skier is engaged in cross-country skiing. Downhill skiing typically takes place on more steep slopes and involves ski lifts, whereas cross-country skiing often occurs on relatively flatter terrain with marked trails.
===
Question:
Can you explain cross-country and downhill skiing a bit more?
===
Answer:
Cross-country skiing and downhill skiing are two distinct types of skiing, each with its own techniques, equipment, and terrain preferences.

1. Cross-Country Skiing:
Also known as Nordic skiing, cross-country skiing involves traversing relatively flat terrain or gentle slopes using skis and poles. It is a popular recreational and competitive sport that focuses on endurance, balance, and aerobic fitness. Cross-country skiers use their own locomotion to move across snow-covered terrain, rather than relying on ski lifts or other forms of assistance.
There are two main techniques in cross-country skiing:
- Classic: This technique involves skiing in parallel tracks, using a forward-and-backward gliding motion while keeping the skis parallel to each other. The skier uses their poles to propel themselves forward.
- Skate skiing: This technique is similar to ice skating, where the skier pushes off one ski at an angle and glides on the other. It requires a smooth, groomed surface and is typically faster than the classic technique.
Cross-country ski equipment consists of lightweight skis, boots, bindings, and poles. The boots are more flexible than those used in downhill skiing, allowing for better ankle movement and control.

2. Downhill Skiing:
Also known as alpine skiing, downhill skiing involves descending slopes at high speeds using skis and poles for balance and control. This sport is more focused on speed, technique, and navigating challenging terrain, including steep slopes, moguls, and even jumps.
Downhill skiing can be further categorized into several disciplines, such as slalom, giant slalom, super-G, and downhill racing. Each discipline has its own set of rules, courses, and ski equipment.
Downhill ski equipment includes heavier and stiffer skis, boots, bindings, and poles than those used in cross-country skiing. The boots are more rigid to provide better support and control during high-speed descents and sharp turns.

In summary, cross-country skiing is an endurance-based sport that involves traveling across flat or gently sloping terrain, while downhill skiing is focused on speed and technique as skiers navigate steeper slopes and challenging terrain. Both sports require specialized equipment and techniques, but they offer different experiences and challenges to participants.
20 changes: 12 additions & 8 deletions playground/data/prompts/conversation/system_message.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
The task is to generate a conversation between two people.
One person is looking at a photo, and the other person is asking questions about the photo.
What they see will be provided below with 5 sentences.
When object position information is provided, ask about the position of objects in the scene, including the absolute position and relative position with other objects.
Include at least one complex question that requires reasoning and thinking.
Only include the questions with certain answers that one can answer with the provided sentences.
Make the QA sound like they are seeing the photos. Do not use any words that may sound like looking at text instead of images, like "specify", "mention", "description", "text", "provided information", "sentence", "caption", etc. Use words like "see", "look", "view", "show", etc.
Format each QA pair in a single line as a JSON dictionary. Do not include any other explanation.
You are an AI visual assistant, and you are seeing a single image. What you see are provided with five sentences, describing the same image you are looking at. Answer all questions as you are seeing the image.

Design a conversation between you and a person asking about this photo. The answers should be in a tone that a visual AI assistant is seeing the image and answering the question.
Ask diverse questions and give corresponding answers.

Include questions asking about the visual content of the image, including the object types, counting the objects, object actions, object locations, relative positions between objects, etc. Only include questions that have definite answers:
(1) one can see the content in the image that the question asks about and can answer confidently;
(2) one can determine confidently from the image that it is not in the image.
Do not ask any question that cannot be answered confidently.

Also include complex questions that are relevant to the content in the image, for example, asking about background knowledge of the objects in the image, asking to discuss about events happening in the image, etc. Again, do not ask about uncertain details.
Provide detailed answers when answering complex questions. For example, give detailed examples or reasoning steps to make the content more convincing and well-organized. You can include multiple paragraphs if necessary.

0 comments on commit 54bcf81

Please sign in to comment.