Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): automatically log http calls to API #3081

Merged
merged 6 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ export declare class NangoAction {
triggerAction<In = unknown, Out = object>(providerConfigKey: string, connectionId: string, actionName: string, input?: In): Promise<Out>;
triggerSync(providerConfigKey: string, connectionId: string, syncName: string, fullResync?: boolean): Promise<void | string>;
private sendLogToPersist;
private logAPICall;
}
export declare class NangoSync extends NangoAction {
lastSyncDate?: Date;
Expand Down
56 changes: 54 additions & 2 deletions packages/shared/lib/sdk/sync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import { validateData } from './dataValidation.js';
import { NangoError } from '../utils/error.js';
import type { DBTeam, GetPublicIntegration, MessageRowInsert } from '@nangohq/types';
import { getProvider } from '../services/providers.js';
import { redactHeaders, redactURL } from '../utils/http.js';

const logger = getLogger('SDK');

Expand Down Expand Up @@ -487,6 +488,13 @@ export class NangoAction {
};
}
}
if (!config.axios?.response) {
axiosSettings.interceptors = {
response: {
onFulfilled: this.logAPICall.bind(this)
}
};
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do I read correctly that it doesn't override the interceptors defined a few lines above that is used for snapshoting in dry-mode?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes exactly, I'll add a comment


if (config.environmentName) {
this.environmentName = config.environmentName;
Expand Down Expand Up @@ -763,7 +771,12 @@ export class NangoAction {
const level = userDefinedLevel?.level ?? 'info';

if (this.dryRun) {
logger[logLevelToLogger[level] ?? 'info'].apply(null, args as any);
// TODO: control this logger inside dryrun instead
if (args.length > 1 && 'type' in args[1] && args[1].type === 'http') {
logger[logLevelToLogger[level] ?? 'info'].apply(null, [args[0], { status: args[1]?.response?.code || 'xxx' }] as any);
} else {
logger[logLevelToLogger[level] ?? 'info'].apply(null, args as any);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would something like this be easier to read

const status = args[1]?.type == 'http' ? args[1].response?.code: undefined
const logArgs =  status ? [args[0], { status }] : args
logger[logLevelToLogger[level] ?? 'info'].apply(null, logArgs as any);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly I don't know ahah
I have split the repetitive part

return;
}

Expand Down Expand Up @@ -912,10 +925,49 @@ export class NangoAction {
}

if (response.status > 299) {
logger.error(`Request to persist API (log) failed: errorCode=${response.status} response='${JSON.stringify(response.data)}'`, this.stringify());
logger.error(
`Request to persist API (log) failed: errorCode=${response.status} response='${JSON.stringify(response.data)}' log=${JSON.stringify(log)}`,
this.stringify()
);
throw new Error(`Failed to log: ${JSON.stringify(response.data)}`);
}
}

private logAPICall(res: AxiosResponse): AxiosResponse {
if (!res.config.url) {
return res;
}

const valuesToFilter: string[] = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small thought for the future: Could we compute our filter array once up front before any runner work and have the logger lean on it across all logging?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed. The valuesToFilter should not changed within a single execution

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually it's changing because connection's credentials can change during execution, but the value could be cached indeed. I'll add a comment

...Array.from(this.memoizedConnections.values()).reduce<string[]>((acc, conn) => {
if (!conn) {
return acc;
}
acc.push(...Object.values(conn.connection.credentials));
return acc;
}, []),
this.nango.secretKey
];

const method = res.config.method?.toLocaleUpperCase(); // axios put it in lowercase;
void this.log(
`${method} ${res.config.url}`,
{
type: 'http',
request: {
method: method,
url: redactURL({ url: res.config.url, valuesToFilter }),
headers: redactHeaders({ headers: res.config.headers })
bodinsamuel marked this conversation as resolved.
Show resolved Hide resolved
},
response: {
code: res.status,
headers: redactHeaders({ headers: res.headers, valuesToFilter })
}
},
{ level: res.status > 299 ? 'error' : 'info' }
);
return res;
}
}

export class NangoSync extends NangoAction {
Expand Down
4 changes: 2 additions & 2 deletions packages/shared/lib/services/proxy.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ class ProxyService {
},
response: {
code: response.status,
headers: response.headers as Record<string, string>
headers: (response.headers || {}) as Record<string, string>
}
}
]
Expand Down Expand Up @@ -574,7 +574,7 @@ class ProxyService {
},
response: {
code: error.response?.status || 500,
headers: error.response?.headers as Record<string, string>
headers: (error.response?.headers || {}) as Record<string, string>
},
error: {
name: error.name,
Expand Down
Loading