Skip to content

Commit

Permalink
Introduce pg_stat_checkpointer
Browse files Browse the repository at this point in the history
Historically, the statistics of the checkpointer have been always part
of pg_stat_bgwriter.  This commit removes a few columns from
pg_stat_bgwriter, and introduces pg_stat_checkpointer with equivalent,
renamed columns (plus a new one for the reset timestamp):
- checkpoints_timed -> num_timed
- checkpoints_req -> num_requested
- checkpoint_write_time -> write_time
- checkpoint_sync_time -> sync_time
- buffers_checkpoint -> buffers_written

The fields of PgStat_CheckpointerStats and its SQL functions are renamed
to match with the new field names, for consistency.  Note that
background writer and checkpointer have been split into two different
processes in commits 806a2ae and bf405ba.  The pgstat
structures were already split, making this change straight-forward.

Bump catalog version.

Author: Bharath Rupireddy
Reviewed-by: Bertrand Drouvot, Andres Freund, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACVxX2ii=66RypXRweZe2EsBRiPMj0aHfRfHUeXJcC7kHg@mail.gmail.com
  • Loading branch information
michaelpq committed Oct 30, 2023
1 parent bf01e1b commit 96f0526
Show file tree
Hide file tree
Showing 14 changed files with 187 additions and 95 deletions.
98 changes: 75 additions & 23 deletions doc/src/sgml/monitoring.sgml
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</entry>
</row>

<row>
<entry><structname>pg_stat_checkpointer</structname><indexterm><primary>pg_stat_checkpointer</primary></indexterm></entry>
<entry>One row only, showing statistics about the
checkpointer process's activity. See
<link linkend="monitoring-pg-stat-checkpointer-view">
<structname>pg_stat_checkpointer</structname></link> for details.
</entry>
</row>

<row>
<entry><structname>pg_stat_database</structname><indexterm><primary>pg_stat_database</primary></indexterm></entry>
<entry>One row per database, showing database-wide statistics. See
Expand Down Expand Up @@ -2868,7 +2877,7 @@ description | Waiting for a newly initialized WAL file to reach durable storage

<para>
The <structname>pg_stat_bgwriter</structname> view will always have a
single row, containing global data for the cluster.
single row, containing data about the background writer of the cluster.
</para>

<table id="pg-stat-bgwriter-view" xreflabel="pg_stat_bgwriter">
Expand All @@ -2888,77 +2897,118 @@ description | Waiting for a newly initialized WAL file to reach durable storage
<tbody>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>checkpoints_timed</structfield> <type>bigint</type>
<structfield>buffers_clean</structfield> <type>bigint</type>
</para>
<para>
Number of scheduled checkpoints that have been performed
Number of buffers written by the background writer
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>checkpoints_req</structfield> <type>bigint</type>
<structfield>maxwritten_clean</structfield> <type>bigint</type>
</para>
<para>
Number of requested checkpoints that have been performed
Number of times the background writer stopped a cleaning
scan because it had written too many buffers
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>checkpoint_write_time</structfield> <type>double precision</type>
<structfield>buffers_alloc</structfield> <type>bigint</type>
</para>
<para>
Total amount of time that has been spent in the portion of
checkpoint processing where files are written to disk, in milliseconds
Number of buffers allocated
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>checkpoint_sync_time</structfield> <type>double precision</type>
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
</para>
<para>
Total amount of time that has been spent in the portion of
checkpoint processing where files are synchronized to disk, in
milliseconds
Time at which these statistics were last reset
</para></entry>
</row>
</tbody>
</tgroup>
</table>

</sect2>

<sect2 id="monitoring-pg-stat-checkpointer-view">
<title><structname>pg_stat_checkpointer</structname></title>

<indexterm>
<primary>pg_stat_checkpointer</primary>
</indexterm>

<para>
The <structname>pg_stat_checkpointer</structname> view will always have a
single row, containing data about the checkpointer process of the cluster.
</para>

<table id="pg-stat-checkpointer-view" xreflabel="pg_stat_checkpointer">
<title><structname>pg_stat_checkpointer</structname> View</title>
<tgroup cols="1">
<thead>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>buffers_checkpoint</structfield> <type>bigint</type>
Column Type
</para>
<para>
Number of buffers written during checkpoints
Description
</para></entry>
</row>
</thead>

<tbody>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>buffers_clean</structfield> <type>bigint</type>
<structfield>num_timed</structfield> <type>bigint</type>
</para>
<para>
Number of buffers written by the background writer
Number of scheduled checkpoints that have been performed
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>maxwritten_clean</structfield> <type>bigint</type>
<structfield>num_requested</structfield> <type>bigint</type>
</para>
<para>
Number of times the background writer stopped a cleaning
scan because it had written too many buffers
Number of requested checkpoints that have been performed
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>buffers_alloc</structfield> <type>bigint</type>
<structfield>write_time</structfield> <type>double precision</type>
</para>
<para>
Number of buffers allocated
Total amount of time that has been spent in the portion of
checkpoint processing where files are written to disk, in milliseconds
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>sync_time</structfield> <type>double precision</type>
</para>
<para>
Total amount of time that has been spent in the portion of
checkpoint processing where files are synchronized to disk, in
milliseconds
</para></entry>
</row>

<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>buffers_written</structfield> <type>bigint</type>
</para>
<para>
Number of buffers written during checkpoints
</para></entry>
</row>

Expand Down Expand Up @@ -4669,8 +4719,10 @@ description | Waiting for a newly initialized WAL file to reach durable storage
Resets some cluster-wide statistics counters to zero, depending on the
argument. The argument can be <literal>bgwriter</literal> to reset
all the counters shown in
the <structname>pg_stat_bgwriter</structname>
view, <literal>archiver</literal> to reset all the counters shown in
the <structname>pg_stat_bgwriter</structname> view,
<literal>checkpointer</literal> to reset all the counters shown in
the <structname>pg_stat_checkpointer</structname> view,
<literal>archiver</literal> to reset all the counters shown in
the <structname>pg_stat_archiver</structname> view,
<literal>io</literal> to reset all the counters shown in the
<structname>pg_stat_io</structname> view,
Expand Down
4 changes: 2 additions & 2 deletions src/backend/access/transam/xlog.c
Original file line number Diff line number Diff line change
Expand Up @@ -6349,8 +6349,8 @@ LogCheckpointEnd(bool restartpoint)
CheckpointStats.ckpt_sync_end_t);

/* Accumulate checkpoint timing summary data, in milliseconds. */
PendingCheckpointerStats.checkpoint_write_time += write_msecs;
PendingCheckpointerStats.checkpoint_sync_time += sync_msecs;
PendingCheckpointerStats.write_time += write_msecs;
PendingCheckpointerStats.sync_time += sync_msecs;

/*
* All of the published timing statistics are accounted for. Only
Expand Down
14 changes: 9 additions & 5 deletions src/backend/catalog/system_views.sql
Original file line number Diff line number Diff line change
Expand Up @@ -1111,16 +1111,20 @@ CREATE VIEW pg_stat_archiver AS

CREATE VIEW pg_stat_bgwriter AS
SELECT
pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints_timed,
pg_stat_get_bgwriter_requested_checkpoints() AS checkpoints_req,
pg_stat_get_checkpoint_write_time() AS checkpoint_write_time,
pg_stat_get_checkpoint_sync_time() AS checkpoint_sync_time,
pg_stat_get_bgwriter_buf_written_checkpoints() AS buffers_checkpoint,
pg_stat_get_bgwriter_buf_written_clean() AS buffers_clean,
pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean,
pg_stat_get_buf_alloc() AS buffers_alloc,
pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;

CREATE VIEW pg_stat_checkpointer AS
SELECT
pg_stat_get_checkpointer_num_timed() AS num_timed,
pg_stat_get_checkpointer_num_requested() AS num_requested,
pg_stat_get_checkpointer_write_time() AS write_time,
pg_stat_get_checkpointer_sync_time() AS sync_time,
pg_stat_get_checkpointer_buffers_written() AS buffers_written,
pg_stat_get_checkpointer_stat_reset_time() AS stats_reset;

CREATE VIEW pg_stat_io AS
SELECT
b.backend_type,
Expand Down
6 changes: 3 additions & 3 deletions src/backend/postmaster/checkpointer.c
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
PendingCheckpointerStats.requested_checkpoints++;
PendingCheckpointerStats.num_requested++;
}

/*
Expand All @@ -372,7 +372,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
PendingCheckpointerStats.timed_checkpoints++;
PendingCheckpointerStats.num_timed++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
Expand Down Expand Up @@ -569,7 +569,7 @@ HandleCheckpointerInterrupts(void)
* updates the statistics, increment the checkpoint request and flush
* out pending statistic.
*/
PendingCheckpointerStats.requested_checkpoints++;
PendingCheckpointerStats.num_requested++;
ShutdownXLOG(0, 0);
pgstat_report_checkpointer();
pgstat_report_wal(true);
Expand Down
2 changes: 1 addition & 1 deletion src/backend/storage/buffer/bufmgr.c
Original file line number Diff line number Diff line change
Expand Up @@ -2751,7 +2751,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
PendingCheckpointerStats.buf_written_checkpoints++;
PendingCheckpointerStats.buffers_written++;
num_written++;
}
}
Expand Down
21 changes: 11 additions & 10 deletions src/backend/utils/activity/pgstat_checkpointer.c
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@ pgstat_report_checkpointer(void)
pgstat_begin_changecount_write(&stats_shmem->changecount);

#define CHECKPOINTER_ACC(fld) stats_shmem->stats.fld += PendingCheckpointerStats.fld
CHECKPOINTER_ACC(timed_checkpoints);
CHECKPOINTER_ACC(requested_checkpoints);
CHECKPOINTER_ACC(checkpoint_write_time);
CHECKPOINTER_ACC(checkpoint_sync_time);
CHECKPOINTER_ACC(buf_written_checkpoints);
CHECKPOINTER_ACC(num_timed);
CHECKPOINTER_ACC(num_requested);
CHECKPOINTER_ACC(write_time);
CHECKPOINTER_ACC(sync_time);
CHECKPOINTER_ACC(buffers_written);
#undef CHECKPOINTER_ACC

pgstat_end_changecount_write(&stats_shmem->changecount);
Expand Down Expand Up @@ -92,6 +92,7 @@ pgstat_checkpointer_reset_all_cb(TimestampTz ts)
&stats_shmem->stats,
sizeof(stats_shmem->stats),
&stats_shmem->changecount);
stats_shmem->stats.stat_reset_timestamp = ts;
LWLockRelease(&stats_shmem->lock);
}

Expand All @@ -113,10 +114,10 @@ pgstat_checkpointer_snapshot_cb(void)

/* compensate by reset offsets */
#define CHECKPOINTER_COMP(fld) pgStatLocal.snapshot.checkpointer.fld -= reset.fld;
CHECKPOINTER_COMP(timed_checkpoints);
CHECKPOINTER_COMP(requested_checkpoints);
CHECKPOINTER_COMP(checkpoint_write_time);
CHECKPOINTER_COMP(checkpoint_sync_time);
CHECKPOINTER_COMP(buf_written_checkpoints);
CHECKPOINTER_COMP(num_timed);
CHECKPOINTER_COMP(num_requested);
CHECKPOINTER_COMP(write_time);
CHECKPOINTER_COMP(sync_time);
CHECKPOINTER_COMP(buffers_written);
#undef CHECKPOINTER_COMP
}
35 changes: 18 additions & 17 deletions src/backend/utils/adt/pgstatfuncs.c
Original file line number Diff line number Diff line change
Expand Up @@ -1182,21 +1182,21 @@ PG_STAT_GET_DBENTRY_FLOAT8_MS(idle_in_transaction_time)
PG_STAT_GET_DBENTRY_FLOAT8_MS(session_time)

Datum
pg_stat_get_bgwriter_timed_checkpoints(PG_FUNCTION_ARGS)
pg_stat_get_checkpointer_num_timed(PG_FUNCTION_ARGS)
{
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->timed_checkpoints);
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->num_timed);
}

Datum
pg_stat_get_bgwriter_requested_checkpoints(PG_FUNCTION_ARGS)
pg_stat_get_checkpointer_num_requested(PG_FUNCTION_ARGS)
{
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->requested_checkpoints);
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->num_requested);
}

Datum
pg_stat_get_bgwriter_buf_written_checkpoints(PG_FUNCTION_ARGS)
pg_stat_get_checkpointer_buffers_written(PG_FUNCTION_ARGS)
{
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_checkpoints);
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buffers_written);
}

Datum
Expand All @@ -1212,19 +1212,25 @@ pg_stat_get_bgwriter_maxwritten_clean(PG_FUNCTION_ARGS)
}

Datum
pg_stat_get_checkpoint_write_time(PG_FUNCTION_ARGS)
pg_stat_get_checkpointer_write_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
PG_RETURN_FLOAT8((double)
pgstat_fetch_stat_checkpointer()->checkpoint_write_time);
pgstat_fetch_stat_checkpointer()->write_time);
}

Datum
pg_stat_get_checkpoint_sync_time(PG_FUNCTION_ARGS)
pg_stat_get_checkpointer_sync_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
PG_RETURN_FLOAT8((double)
pgstat_fetch_stat_checkpointer()->checkpoint_sync_time);
pgstat_fetch_stat_checkpointer()->sync_time);
}

Datum
pg_stat_get_checkpointer_stat_reset_time(PG_FUNCTION_ARGS)
{
PG_RETURN_TIMESTAMPTZ(pgstat_fetch_stat_checkpointer()->stat_reset_timestamp);
}

Datum
Expand Down Expand Up @@ -1684,14 +1690,9 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS)
if (strcmp(target, "archiver") == 0)
pgstat_reset_of_kind(PGSTAT_KIND_ARCHIVER);
else if (strcmp(target, "bgwriter") == 0)
{
/*
* Historically checkpointer was part of bgwriter, continue to reset
* both for now.
*/
pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER);
else if (strcmp(target, "checkpointer") == 0)
pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER);
}
else if (strcmp(target, "io") == 0)
pgstat_reset_of_kind(PGSTAT_KIND_IO);
else if (strcmp(target, "recovery_prefetch") == 0)
Expand All @@ -1702,7 +1703,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\", \"bgwriter\", \"io\", \"recovery_prefetch\", or \"wal\".")));
errhint("Target must be \"archiver\", \"bgwriter\", \"checkpointer\", \"io\", \"recovery_prefetch\", or \"wal\".")));

PG_RETURN_VOID();
}
Expand Down
2 changes: 1 addition & 1 deletion src/include/catalog/catversion.h
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,6 @@
*/

/* yyyymmddN */
#define CATALOG_VERSION_NO 202310271
#define CATALOG_VERSION_NO 202310301

#endif
Loading

0 comments on commit 96f0526

Please sign in to comment.