第4次更新

1、优化prompt;
2、解决日程删除错误；
3、iotm demo tool整体更新。

README.md

@@ -4,10 +4,14 @@
     <br>
     <img src="images/icon.png" alt="Fay">
     <h1>Fay数字人 AI Agent版</h1>
-    “agent”即代理，它能够代替你完成决策规划并执行，这一切都依赖目前最强的大语言模型的ReAct能力。
+    “agent”即代理，它能够代替你完成决策规划并执行，这一切都依赖目前最强的大语言模型的ReAct能力。不同于助理版的一问一答，agent版的Fay可以实现自动代理执行的同时，在它认为必要时候会触发数字人或者直接的声音输出。
 </div>
 
-**12月迟来的报到，Fay数字人 AI Agent版（含智慧农业应用demo）第3版正式上传！**
+
+
+
+
+**12月迟来的报到，Fay数字人 AI Agent版（含智慧农业箱的操作demo代码，如果你需要完整代码可以公众号留言申请获取）第4版正式上传！**
 
 如果你需要是一个线上线下的销售员，请移步[`带货完整版`](https://github.com/TheRamU/Fay/tree/fay-sales-edition)                       
 
@@ -53,6 +57,8 @@ python main.py
 
 + 启动数字人[xszyou/fay-ue5: 可对接fay数字人的ue5工程 (github.com)](https://github.com/xszyou/fay-ue5)
 
+
+
 ### **联系**
 
 **商务QQ: 467665317**

README_EN.md

@@ -0,0 +1,65 @@
+[`中文`](https://github.com/TheRamU/Fay/blob/main/README.md)
+
+<div align="center">
+    <br>
+    <img src="images/icon.png" alt="Fay">
+    <h1>Fay Digital Human AI Agent Version</h1>
+    An "agent" is a representative that can make decisions and execute plans for you, relying on the powerful ReAct capability of the most advanced large language models.
+</div>
+
+
+
+
+
+**Belated December announcement, the 4th edition of Fay Digital Human AI Agent Version (complete code for smart agriculture box can be requested via our public channel) is officially uploaded!**
+
+If you need an online and offline salesperson, please go to [`Complete Retail Version`](https://github.com/TheRamU/Fay/tree/fay-sales-edition)
+
+If you need a digital human assistant for human-computer interaction (and yes, you can command it to switch devices on and off), please go to [`Complete Assistant Version`](https://github.com/TheRamU/Fay/tree/fay-assistant-edition)
+
+***“Exceptional products deserve to be reimagined with digital humans”***
+
+Highlights: Proactive execution of planned tasks without the need for question-and-answer interactions, automatic planning and use of the agent tool to complete tasks; use of OpenAI TTS; use of a vector database for permanent memory and memory retrieval;
+
+![](images/agent_demo.gif)
+
+​                                                                       (Above image: Testing ReAct capabilities）
+
+## **Installation Instructions**
+
+### **System Requirements** 
+
+- Python 3.9, 3.10
+- Windows, macOS, Linux
+
+### **Installing Dependencies**
+
+```shell
+pip install -r requirements.txt
+```
+
+### **Configuring Application Keys**
+
++ Enter your GPT-4 key in `./system.conf` 
+
+### **Launching the Controller**
+
+Start the Fay controller
+
+```shell
+python main.py
+```
+
+### **Launching the Digital Human (Optional)**
+
++ Launch the digital human[xszyou/fay-ue5: 可对接fay数字人的ue5工程 (github.com)](https://github.com/xszyou/fay-ue5)
+
+
+
+### **Contact**
+
+**Business QQ: 467665317**
+
+Join the discussion group by following the public account Fay Digital Human (please star this repository first)
+
+<img src="images/2.jpg"  />

WebSocket.md

@@ -1,3 +1,4 @@
+#这是与数字人通讯的接口说明
 ## 消息格式
 
 通讯地址: [`ws://127.0.0.1:10002`](ws://127.0.0.1:10002)

agent/agent_service.py

@@ -51,7 +51,7 @@ def parse_repeat_rule(rule, task_time):
 
 # 执行任务
 def execute_task(task_time, id, content):
-    content = content
+    content = "执行任务->现在" + content
     agent.run(content)
     del scheduled_tasks[id]
     # 如果不重复，执行后删除记录
@@ -96,19 +96,17 @@ def agent_start():
 
     #初始计划
     agent.run("""执行任务-->
-        请为我一个个时间设置初始计划：
-        1、在每天12:30到13:30安静不影响主人休息;
-        2、每天12点提醒主人吃饭;  
-        3、在星期一到星期五13:30提醒主人开始工作; 
-        4、在星期一到星期五15:15提醒主人冲咖啡; 
-        5、在星期一、星期三的11:15提醒主人开会; 
-        6、在星期五17:30提醒主人开会;
-        7、在星期一到星期五18:00提醒主人下班;
+        你是一个智慧农业实验箱里的ai，你的责任是陪伴主人生活、工作，以及协助主人打理好农业种植箱里的农作物：
+        1、在每天早上8点、中午12点、晚上10点检查农业种植箱的状态是否附合设定的预期执行，如果不符合请告知我调整。
+        2、每天12点“语音提醒主人吃饭”;  
+        3、在星期一到星期五13:30“语音提醒主人开始工作”; 
+        4、在星期一到星期五15:15“语音提醒主人冲咖啡”; 
+        5、在星期一、星期三的11:15“语音提醒主人开会”; 
+        6、在星期五17:30“语音提醒主人开会”;
+        7、在星期一到星期五18:00“语音提醒主人下班”;
         8、在每天21点陪主人聊聊天;  
-        9、在每天晚上10:30会跟据天气预报信息和前一天的运行情况，检查iotm系统当天的控制规则；
-        10、在每天早上8点、中午12点、晚上10点检查农业种植箱的状态是否附合设定的预期执行，
-        如果不符合请告知我调整。
-        另外：在任何意外的情况需要进一步指导请马联系我。
+        9、在每天晚上10:30会跟据第二天的天气预报信息和当天的运行情况，检查iotm系统当天的控制规则；
+
         """)
 
 def agent_stop():

agent/fay_agent.py

@@ -17,6 +17,8 @@
 from agent.tools.Say import Say
 from agent.tools.QueryTimerDB import QueryTimerDB
 from agent.tools.DeleteTimer import DeleteTimer
+from agent.tools.GetSwitchLog import GetSwitchLog
+from agent.tools.getOnRunLinkage import getOnRunLinkage
 
 import utils.config_util as utils
 from core.content_db import Content_Db
@@ -35,7 +37,7 @@ def __init__(self):
         embedding_fn = OpenAIEmbeddings()
 
         #创建llm
-        llm = ChatOpenAI(verbose=True)#model="gpt-4-1106-preview"
+        llm = ChatOpenAI(model="gpt-4-1106-preview", verbose=True)
 
         #创建向量数据库
         vectorstore = FAISS(embedding_fn, index, InMemoryDocstore({}), {})
@@ -60,6 +62,9 @@ def __init__(self):
         say_tool = Say()
         query_timer_db_tool = QueryTimerDB()
         delete_timer_tool = DeleteTimer()
+        get_switch_log = GetSwitchLog()
+        get_on_run_linkage = getOnRunLinkage()
+
         tools = [
             Tool(
                 name=my_timer.name,
@@ -111,6 +116,16 @@ def __init__(self):
                 func=delete_timer_tool.run,
                 description=delete_timer_tool.description
             ),
+            Tool(
+                name=get_switch_log.name,
+                func=get_switch_log.run,
+                description=get_switch_log.description
+            ),
+            Tool(
+                name=get_on_run_linkage.name,
+                func=get_on_run_linkage.run,
+                description=get_on_run_linkage.description
+            ),
         ]
 
 
@@ -126,7 +141,7 @@ def run(self, input_text):
         wsa_server.get_web_instance().add_cmd({"panelReply": {"type":"member","content":input_text.replace('主人语音说了：', '').replace('主人文字说了：', '')}})
         result = None
         try:
-            result = self.agent.run(input_text.replace('执行任务-->', ''))
+            result = self.agent.run(input_text)
         except Exception as e:
             print(e)
         result = "执行完毕" if result is None or result == "N/A" else result

agent/tools/CheckSensor.py

@@ -3,11 +3,13 @@
 
 import requests
 from langchain.tools import BaseTool
-
+import time
+import agent.tools.IotmService as IotmService
+from datetime import datetime
 
 class CheckSensor(BaseTool):
     name = "CheckSensor"
-    description = "此工具用于查询农业箱传感器数据及设备开关状态"
+    description = "此工具用于查询农业箱在线状态、传感器数据、设备开关状态"
 
     def __init__(self):
         super().__init__()
@@ -18,73 +20,58 @@ async def _arun(self, *args: Any, **kwargs: Any) -> Any:
 
 
     def _run(self, para: str) -> str:
-        return """
-    {
-    "result": True,
-    "ts": "2023-05-09 17:54:31.948",
-    "data": [
-        "co2": 
-            {
-                "ts": "2022-05-09 17:54:31.948",
-                "val": "1000ppm",
-        "desc":"箱内的二氧化碳含量"
-                },
-
-        "inside_temperature": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": 28,
-        "desc":"箱内的温度"
-                },
-
-        "inside_humidity": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": 80,
-        "desc":"箱内的湿度"
-                },
-
-        "outside_temperature": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": 28,
-        "desc":"箱外的温度"
-                },
-
-        "outside_humidity": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": 80,
-        "desc":"箱外的湿度"
-                },
-
-        "inside_illuminance": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": "300lux"
-        "desc":"箱内的光照强度的值，当箱内光照强度太低时，生长灯会被打开，传感器位置是可以检测到生长灯的亮度的"
-                },
-
-        "inside_soil": 
-                {
-                    "ts": "2022-05-09 17:54:31.948",
-                    "val": 70
-        "desc":"箱内土壤的湿度，检测的数所有延迟，水在土壤里有个渗透的过程"
-                },
-
-
-            
-        ],
-        "制冷":"off",
-        "加热":"off",
-        "通风":"off",
-        "加co2":"off",
-        "补光":"off",
-        "浇水":"off"
-    }
-"""
-
-
+        #箱子信息
+        building_infos = IotmService.get_building_unit()
+        is_online = building_infos.get('isonline', 0)
+        #传感器数据
+        sensor_all_infos = IotmService.get_latest_list()
+        sensor_infos = sensor_all_infos['data']
+        desc_list = {
+        'temperature': '温度',
+        'humidity': '湿度',
+        'co2': '二氧化碳',
+        'light': '箱内的光照强度的值，当箱内光照强度太低时，生长灯会被打开，传感器位置是可以检测到生长灯的亮度的',
+        'air': '污染气体',
+        'nh3': '氨气'
+        }
+
+        infos = []
+        for sensor_type, sensor_data in sensor_infos.items():
+                for data_point in sensor_data:
+                        if sensor_type == 'temperature':
+                             if data_point['port'] == 'MP14':
+                                  description = '箱外温度'
+                             else:
+                                  description = '箱内温度' 
+
+                        elif sensor_type == 'humidity':
+                             if data_point['port'] == 'MP14':
+                                  description = '箱外湿度'
+                             elif data_point['port'] == 'S34':
+                                  description = '箱内土壤的湿度，检测的数所有延迟，水在土壤里有个渗透的过程'
+                             else:
+                                  description = '箱内湿度' 
+                        else:
+                                description = desc_list.get(sensor_type, 'Unknown')  # Get description from desc_list, default to 'Unknown'
+                        timestamp = data_point['ts']
+                        value = data_point['val']
+                        infos.append({'ts': timestamp, 'val': value, 'desc':description })            
+        #开关数据
+        switch_all_infos = IotmService.get_switch_info()
+        switch_infos = {}
+        switch_dict = switch_all_infos[0]
+        #设备配置
+        switch_infos['小风扇'] = 'on' if switch_dict.get('onoff1', '') == '1' else 'off'
+        switch_infos['电热风扇'] = 'on' if switch_dict.get('onoff2', '') == '1' else 'off'
+        switch_infos['制冷风扇'] = 'on' if switch_dict.get('onoff3', '') == '1' else 'off'
+        switch_infos['水开关'] = 'on' if switch_dict.get('onoff4', '') == '1' else 'off'
+        switch_infos['肥料开关'] = 'on' if switch_dict.get('onoff5', '') == '1' else 'off'
+        switch_infos['植物生长灯'] = 'on' if switch_dict.get('onoff6', '') == '1' else 'off'
+        switch_infos['二氧化碳'] = 'on' if switch_dict.get('onoff7', '') == '1' else 'off'
+        current_time = datetime.now()
+        current_time_str = current_time.strftime("%Y-%m-%d %H:%M:%S")
+        result = {'sensor_infos': infos, 'switch_infos': switch_infos, 'is_online': is_online, 'ts' : current_time_str} 
+        return result
 
 if __name__ == "__main__":
     tool = CheckSensor()

agent/tools/DeleteTimer.py

@@ -1,36 +1,36 @@
-import abc
 import sqlite3
 from typing import Any
-import ast
 
 from langchain.tools import BaseTool
 
 from agent import agent_service
 
 
-class DeleteTimer(BaseTool, abc.ABC):
+class DeleteTimer(BaseTool):
     name = "DeleteTimer"
-    description = "用于删除某一个定时任务，接受任务id作为参数，如：('2')"
+    description = "用于删除某一个日程，接受任务id作为参数，如：2"
 
     def __init__(self):
         super().__init__()
 
-    async def _arun(self, *args: Any, **kwargs: Any) -> Any:
-        # 用例中没有用到 arun 不予具体实现
-        pass
-
-
     def _run(self, para) -> str:
-        para = ast.literal_eval(para)
-
-        del agent_service.scheduled_tasks[int(para[0])]
-        conn = sqlite3.connect('timer.db')
-        cursor = conn.cursor()
-        cursor.execute(f"DELETE FROM timer WHERE id = {id}")
-        conn.commit()
-        conn.close()
-
-        return f"{id}任务取消成功"
+        try:
+            id = int(para)
+        except ValueError:
+            return "输入的 ID 无效，必须是数字。"
+
+        if id in agent_service.scheduled_tasks:
+            del agent_service.scheduled_tasks[id]
+
+        try:
+            with sqlite3.connect('timer.db') as conn:
+                cursor = conn.cursor()
+                cursor.execute("DELETE FROM timer WHERE id = ?", (id,))
+                conn.commit()
+        except sqlite3.Error as e:
+            return f"数据库错误: {e}"
+
+        return f"任务 {id} 取消成功。"
 
 
 if __name__ == "__main__":