Support tracing logs query for debugging and add banyandb case for lo…

…gs e2e. (apache#12424)

.github/workflows/skywalking.yaml

@@ -434,6 +434,8 @@ jobs:
           - name: Log ES 8.8.1 Shardng
             config: test/e2e-v2/cases/log/es/es-sharding/e2e.yaml
             env: ES_VERSION=8.8.1
+          - name: Log BanyanDB
+            config: test/e2e-v2/cases/log/banyandb/e2e.yaml
 
           - name: Log FluentBit ES 7.16.3
             config: test/e2e-v2/cases/log/fluent-bit/e2e.yaml

docs/en/api/query-protocol.md

@@ -133,7 +133,7 @@ enum ExpressionResultType {
 extend type Query {
     # Return true if the current storage implementation supports fuzzy query for logs.
     supportQueryLogsByKeywords: Boolean!
-    queryLogs(condition: LogQueryCondition): Logs
+    queryLogs(condition: LogQueryCondition, debug: Boolean): Logs
     # Test the logs and get the results of the LAL output.
     test(requests: LogTestRequest!): LogTestResponse!
     # Read the list of searchable keys

docs/en/changes/changes.md

@@ -29,6 +29,7 @@
 * Polish BanyanDB group and schema creation logic to fix the schema creation failure issue in distributed race conditions.
 * Support tracing topology query for debugging.
 * Fix expression of graph `Current QPS` in MySQL dashboard.
+* Support tracing logs query for debugging.
 
 #### UI
 * Highlight search log keywords.

docs/en/debugging/query-tracing.md

@@ -425,7 +425,6 @@ debuggingTrace:
 
 - Example
 ```shell
-
 curl -X GET 'http://127.0.0.1:12800/debugging/query/topology/getProcessTopology?startTime=2024-07-03&endTime=2024-07-03&step=DAY&service=mock_a_service&serviceLayer=GENERAL&instance=mock_a_service_instance'
 ```
 
@@ -440,6 +439,45 @@ debuggingTrace:
 ...
 ```
 
+### Tracing Log Query
+
+#### Tracing SkyWalking API queryLogs
+ URL: HTTP GET `http://{core restHost}:{core restPort}/debugging/query/log/queryLogs?{parameters}`.
+ Parameters
+
+  | Field                      | Description                                                    | Required                       |
+  |----------------------------|----------------------------------------------------------------|--------------------------------|
+  | startTime                  | The start time of the query                                    | Yes, unless traceId not empty  |               
+  | endTime                    | The end time of the query                                      | Yes, unless traceId not empty  |               
+  | step                       | The query step                                                 | Yes, unless traceId not empty  |  
+  | service                    | The service name                                               | No, require serviceLayer       |               
+  | serviceLayer               | The service layer name                                         | No                             |  
+  | serviceInstance            | The service instance name                                      | No, require service            |               
+  | endpoint                   | The endpoint name                                              | No, require service            |               
+  | traceId                    | The trace ID                                                   | No                             |               
+  | segmentId                  | The segment ID                                                 | No, require traceId            |
+  | spanId                     | The span ID                                                    | No, require traceId            | 
+  | queryOrder                 | The order of the query result, `ASC`, `DES`                    | No, default `DES`              |               
+  | tags                       | The tags of the trace, `key1=value1,key2=value2`               | No                             |  
+  | pageNum                    | The page number of the query result                            | Yes                            |
+  | pageSize                   | The page size of the query result                              | Yes                            |
+  | keywordsOfContent          | The keywords of the log content, `keyword1,keyword2`           | No                             |
+  | excludingKeywordsOfContent | The excluding keywords of the log content, `keyword1,keyword2` | No                             |
+
+- Example
+```shell
+curl -X GET 'http://127.0.0.1:12800/debugging/query/log/queryLogs?service=e2e-service-provider&serviceLayer=GENERAL&startTime=2024-07-09&endTime=2024-07-09&step=DAY&pageNum=1&pageSize=15&queryOrder=ASC&tags=level%3DINFO'
+```
+
+Response will include query result and the debuggingTrace information, the debuggingTrace information is the same as the MQE query tracing:
+
+```yaml
+logs:
+...
+debuggingTrace:
+...
+```
+
 ## Debugging with GraphQL bundled
 When querying the metrics though the [GraphQL APIs](../api/query-protocol.md),
 the query tracing service is also provided within the GraphQL bundled.
@@ -717,3 +755,27 @@ type ProcessTopology {
 - Example
 Same as the MQE query tracing, follow the GraphQL protocol and grammar to query the result and get debuggingTrace information,
 just enable the debug parameter to true.
+
+### Tracing Log Query
+- Bundle API: [Log](../api/query-protocol.md#logs)
+
+```graphql
+extend type Query {
+...
+    queryLogs(condition: LogQueryCondition, debug: Boolean): Logs
+...
+}
+```
+
+```graphql
+type Logs {
+    # When this field is not empty, frontend should display it in UI
+    errorReason: String
+    logs: [Log!]!
+    debuggingTrace: DebuggingTrace
+}
+```
+
+- Example
+Same as the MQE query tracing, follow the GraphQL protocol and grammar to query the result and get debuggingTrace information,
+just enable the debug parameter to true.

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/LogQueryService.java

@@ -22,6 +22,8 @@
 import java.util.List;
 import java.util.stream.Collectors;
 import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingSpan;
+import org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingTraceContext;
 import org.apache.skywalking.oap.server.library.util.StringUtil;
 import org.apache.skywalking.oap.server.core.analysis.IDManager;
 import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
@@ -35,6 +37,7 @@
 import org.apache.skywalking.oap.server.library.module.Service;
 
 import static java.util.Objects.nonNull;
+import static org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingTraceContext.TRACE_CONTEXT;
 
 public class LogQueryService implements Service {
 
@@ -66,6 +69,45 @@ public Logs queryLogs(String serviceId,
                           final List<Tag> tags,
                           List<String> keywordsOfContent,
                           List<String> excludingKeywordsOfContent) throws IOException {
+        DebuggingTraceContext traceContext = TRACE_CONTEXT.get();
+        DebuggingSpan span = null;
+        try {
+            if (traceContext != null) {
+                StringBuilder msg = new StringBuilder();
+                span = traceContext.createSpan("Query Service: queryLogs");
+                msg.append("ServiceId: ").append(serviceId).append(", ");
+                msg.append("ServiceInstanceId: ").append(serviceInstanceId).append(", ");
+                msg.append("EndpointId: ").append(endpointId).append(", ");
+                msg.append("RelatedTrace: ").append(relatedTrace).append(", ");
+                msg.append("Pagination: ").append(paging).append(", ");
+                msg.append("QueryOrder: ").append(queryOrder).append(", ");
+                msg.append("Duration: ").append(duration).append(", ");
+                msg.append("Tags: ").append(tags).append(", ");
+                msg.append("KeywordsOfContent: ").append(keywordsOfContent).append(", ");
+                msg.append("ExcludingKeywordsOfContent: ").append(excludingKeywordsOfContent);
+                span.setMsg(msg.toString());
+            }
+            return invokeQueryLogs(
+                serviceId, serviceInstanceId, endpointId, relatedTrace, paging, queryOrder, duration, tags,
+                keywordsOfContent, excludingKeywordsOfContent
+            );
+        } finally {
+            if (traceContext != null) {
+                traceContext.stopSpan(span);
+            }
+        }
+    }
+
+    private Logs invokeQueryLogs(String serviceId,
+                          String serviceInstanceId,
+                          String endpointId,
+                          TraceScopeCondition relatedTrace,
+                          Pagination paging,
+                          Order queryOrder,
+                          final Duration duration,
+                          final List<Tag> tags,
+                          List<String> keywordsOfContent,
+                          List<String> excludingKeywordsOfContent) throws IOException {
         PaginationUtils.Page page = PaginationUtils.INSTANCE.exchange(paging);
 
         if (nonNull(keywordsOfContent)) {
@@ -79,7 +121,7 @@ public Logs queryLogs(String serviceId,
                                                                    .collect(Collectors.toList());
         }
 
-        Logs logs = getLogQueryDAO().queryLogs(serviceId,
+        Logs logs = getLogQueryDAO().queryLogsDebuggable(serviceId,
                                                serviceInstanceId,
                                                endpointId,
                                                relatedTrace,

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/TraceQueryService.java

@@ -150,6 +150,7 @@ public Trace queryTrace(final String traceId) throws IOException {
                 StringBuilder msg = new StringBuilder();
                 span = traceContext.createSpan("Query Service: queryTrace");
                 msg.append("Condition: TraceId: ").append(traceId);
+                span.setMsg(msg.toString());
             }
             return invokeQueryTrace(traceId);
         } finally {

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/input/LogQueryCondition.java

@@ -21,12 +21,14 @@
 import java.util.List;
 import lombok.Getter;
 import lombok.Setter;
+import lombok.ToString;
 import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
 import org.apache.skywalking.oap.server.core.query.enumeration.Order;
 import org.apache.skywalking.oap.server.core.query.type.Pagination;
 
 @Getter
 @Setter
+@ToString
 public class LogQueryCondition {
     private String serviceId;
     private String serviceInstanceId;

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/input/TraceScopeCondition.java

@@ -19,9 +19,11 @@
 
 import lombok.Getter;
 import lombok.Setter;
+import lombok.ToString;
 
 @Setter
 @Getter
+@ToString
 public class TraceScopeCondition {
 
     private String traceId;

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Logs.java

@@ -24,13 +24,15 @@
 
 import java.util.ArrayList;
 import java.util.List;
+import org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingTrace;
 
 @Setter
 @Getter
 @Accessors(chain = true)
 public class Logs {
     private final List<Log> logs;
     private String errorReason;
+    private DebuggingTrace debuggingTrace;
 
     public Logs() {
         this.logs = new ArrayList<>();

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/query/ILogQueryDAO.java

@@ -31,14 +31,59 @@
 import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
 import org.apache.skywalking.oap.server.core.query.type.KeyValue;
 import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingSpan;
+import org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingTraceContext;
 import org.apache.skywalking.oap.server.library.module.Service;
 
+import static org.apache.skywalking.oap.server.core.query.type.debugging.DebuggingTraceContext.TRACE_CONTEXT;
+
 public interface ILogQueryDAO extends Service {
 
     default boolean supportQueryLogsByKeywords() {
         return false;
     }
 
+    default Logs queryLogsDebuggable(String serviceId,
+                                     String serviceInstanceId,
+                                     String endpointId,
+                                     TraceScopeCondition relatedTrace,
+                                     Order queryOrder,
+                                     int from,
+                                     int limit,
+                                     final Duration duration,
+                                     final List<Tag> tags,
+                                     final List<String> keywordsOfContent,
+                                     final List<String> excludingKeywordsOfContent) throws IOException {
+        DebuggingTraceContext traceContext = TRACE_CONTEXT.get();
+        DebuggingSpan span = null;
+        try {
+            if (traceContext != null) {
+                span = traceContext.createSpan("Query Dao: queryLogs");
+                StringBuilder msg = new StringBuilder();
+                msg.append("ServiceId: ").append(serviceId)
+                   .append(", ServiceInstanceId: ").append(serviceInstanceId)
+                   .append(", EndpointId: ").append(endpointId)
+                   .append(", RelatedTrace: ").append(relatedTrace)
+                   .append(", QueryOrder: ").append(queryOrder)
+                   .append(", From: ").append(from)
+                   .append(", Limit: ").append(limit)
+                   .append(", Duration: ").append(duration)
+                   .append(", Tags: ").append(tags)
+                   .append(", KeywordsOfContent: ").append(keywordsOfContent)
+                   .append(", ExcludingKeywordsOfContent: ").append(excludingKeywordsOfContent);
+                span.setMsg(msg.toString());
+            }
+            return queryLogs(
+                serviceId, serviceInstanceId, endpointId, relatedTrace, queryOrder, from, limit, duration, tags,
+                keywordsOfContent, excludingKeywordsOfContent
+            );
+        } finally {
+            if (traceContext != null && span != null) {
+                traceContext.stopSpan(span);
+            }
+        }
+    }
+
     Logs queryLogs(String serviceId,
                    String serviceInstanceId,
                    String endpointId,