operators-and-hooks-ref.rst

Operators and Hooks Reference

Content

• Fundamentals
• ASF: Apache Software Foundation
• Azure: Microsoft Azure
• AWS: Amazon Web Services
• Google
• Yandex.Cloud
• Service integrations
• Software integrations
• Protocol integrations

Fundamentals

Base:

Module	Guides
:mod:`airflow.hooks.base_hook`
:mod:`airflow.hooks.dbapi_hook`
:mod:`airflow.models.baseoperator`
:mod:`airflow.sensors.base_sensor_operator`

Operators:

Operators	Guides
:mod:`airflow.operators.branch_operator`
:mod:`airflow.operators.check_operator`
:mod:`airflow.operators.dagrun_operator`
:mod:`airflow.operators.dummy_operator`
:mod:`airflow.operators.generic_transfer`
:mod:`airflow.operators.latest_only_operator`
:mod:`airflow.operators.subdag_operator`
:mod:`airflow.operators.sql_branch_operator`

Sensors:

Sensors	Guides
:mod:`airflow.sensors.weekday_sensor`
:mod:`airflow.sensors.external_task_sensor`	:doc:`How to use <howto/operator/external_task_sensor>`
:mod:`airflow.sensors.sql_sensor`
:mod:`airflow.sensors.time_delta_sensor`
:mod:`airflow.sensors.time_sensor`

ASF: Apache Software Foundation

Airflow supports various software created by Apache Software Foundation.

Software operators and hooks

These integrations allow you to perform various operations within software developed by Apache Software Foundation.

Service name	Guides	Hook	Operator	Sensor
Apache Cassandra		:mod:`airflow.providers.apache.cassandra.hooks.cassandra`		:mod:`airflow.providers.apache.cassandra.sensors.record`, :mod:`airflow.providers.apache.cassandra.sensors.table`
Apache Druid		:mod:`airflow.providers.apache.druid.hooks.druid`	:mod:`airflow.providers.apache.druid.operators.druid`, :mod:`airflow.providers.apache.druid.operators.druid_check`
Apache Hive		:mod:`airflow.providers.apache.hive.hooks.hive`	:mod:`airflow.providers.apache.hive.operators.hive`, :mod:`airflow.providers.apache.hive.operators.hive_stats`	:mod:`airflow.providers.apache.hive.sensors.named_hive_partition`, :mod:`airflow.providers.apache.hive.sensors.hive_partition`, :mod:`airflow.providers.apache.hive.sensors.metastore_partition`
Apache Livy		:mod:`airflow.providers.apache.livy.hooks.livy`	:mod:`airflow.providers.apache.livy.operators.livy`	:mod:`airflow.providers.apache.livy.sensors.livy`
Apache Pig		:mod:`airflow.providers.apache.pig.hooks.pig`	:mod:`airflow.providers.apache.pig.operators.pig`
Apache Pinot		:mod:`airflow.providers.apache.pinot.hooks.pinot`
Apache Spark	:doc:`How to use <howto/operator/apache/spark>`	:mod:`airflow.providers.apache.spark.hooks.spark_jdbc`, :mod:`airflow.providers.apache.spark.hooks.spark_jdbc_script`, :mod:`airflow.providers.apache.spark.hooks.spark_sql`, :mod:`airflow.providers.apache.spark.hooks.spark_submit`	:mod:`airflow.providers.apache.spark.operators.spark_jdbc`, :mod:`airflow.providers.apache.spark.operators.spark_sql`, :mod:`airflow.providers.apache.spark.operators.spark_submit`
Apache Sqoop		:mod:`airflow.providers.apache.sqoop.hooks.sqoop`	:mod:`airflow.providers.apache.sqoop.operators.sqoop`
Hadoop Distributed File System (HDFS)		:mod:`airflow.providers.apache.hdfs.hooks.hdfs`		:mod:`airflow.providers.apache.hdfs.sensors.hdfs`
WebHDFS		:mod:`airflow.providers.apache.hdfs.hooks.webhdfs`		:mod:`airflow.providers.apache.hdfs.sensors.web_hdfs`

Transfer operators and hooks

These integrations allow you to copy data from/to software developed by Apache Software Foundation.

Source	Destination	Guide	Operator
Amazon Simple Storage Service (S3)	Apache Hive		:mod:`airflow.providers.apache.hive.operators.s3_to_hive`
Amazon Simple Storage Service (S3)	MySQL		:mod:`airflow.providers.mysql.operators.s3_to_mysql`
Apache Cassandra	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.cassandra_to_gcs`
Apache Hive	Amazon DynamoDB		:mod:`airflow.providers.amazon.aws.operators.hive_to_dynamodb`
Apache Hive	Apache Druid		:mod:`airflow.providers.apache.druid.operators.hive_to_druid`
Apache Hive	MySQL		:mod:`airflow.providers.apache.hive.operators.hive_to_mysql`
Apache Hive	Samba		:mod:`airflow.providers.apache.hive.operators.hive_to_samba`
Microsoft SQL Server (MSSQL)	Apache Hive		:mod:`airflow.providers.apache.hive.operators.mssql_to_hive`
MySQL	Apache Hive		:mod:`airflow.providers.apache.hive.operators.mysql_to_hive`
Vertica	Apache Hive		:mod:`airflow.providers.apache.hive.operators.vertica_to_hive`

Azure: Microsoft Azure

Airflow has limited support for Microsoft Azure.

Service operators and hooks

These integrations allow you to perform various operations within the Microsoft Azure.

Service name	Hook	Operator	Sensor
Azure Batch	:mod:`airflow.providers.microsoft.azure.hooks.azure_batch`	:mod:`airflow.providers.microsoft.azure.operators.azure_batch`
Azure Blob Storage	:mod:`airflow.providers.microsoft.azure.hooks.wasb`	:mod:`airflow.providers.microsoft.azure.operators.wasb_delete_blob`	:mod:`airflow.providers.microsoft.azure.sensors.wasb`
Azure Container Instances	:mod:`airflow.providers.microsoft.azure.hooks.azure_container_instance`, :mod:`airflow.providers.microsoft.azure.hooks.azure_container_registry`, :mod:`airflow.providers.microsoft.azure.hooks.azure_container_volume`	:mod:`airflow.providers.microsoft.azure.operators.azure_container_instances`
Azure Cosmos DB	:mod:`airflow.providers.microsoft.azure.hooks.azure_cosmos`	:mod:`airflow.providers.microsoft.azure.operators.azure_cosmos`	:mod:`airflow.providers.microsoft.azure.sensors.azure_cosmos`
Azure Data Lake Storage	:mod:`airflow.providers.microsoft.azure.hooks.azure_data_lake`	:mod:`airflow.providers.microsoft.azure.operators.adls_list`
Azure Data Explorer	:mod:`airflow.providers.microsoft.azure.hooks.adx`	:mod:`airflow.providers.microsoft.azure.operators.adx`
Azure Files	:mod:`airflow.providers.microsoft.azure.hooks.azure_fileshare`

Transfer operators and hooks

These integrations allow you to copy data from/to Microsoft Azure.

Source	Destination	Guide	Operator
Azure Data Lake Storage	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.adls_to_gcs`
Local	Azure Blob Storage		:mod:`airflow.providers.microsoft.azure.operators.file_to_wasb`
Oracle	Azure Data Lake Storage		:mod:`airflow.providers.microsoft.azure.operators.oracle_to_azure_data_lake_transfer`

AWS: Amazon Web Services

Airflow has support for Amazon Web Services.

All hooks are based on :mod:`airflow.providers.amazon.aws.hooks.base_aws`.

Service operators and hooks

These integrations allow you to perform various operations within the Amazon Web Services.

Service name	Guide	Hook	Operator	Sensor
AWS Batch		:mod:`airflow.providers.amazon.aws.hooks.batch_client`, :mod:`airflow.providers.amazon.aws.hooks.batch_waiters`	:mod:`airflow.providers.amazon.aws.operators.batch`
AWS DataSync	:doc:`How to use <howto/operator/amazon/aws/datasync>`	:mod:`airflow.providers.amazon.aws.hooks.datasync`	:mod:`airflow.providers.amazon.aws.operators.datasync`
AWS Glue Catalog		:mod:`airflow.providers.amazon.aws.hooks.glue_catalog`		:mod:`airflow.providers.amazon.aws.sensors.glue_catalog_partition`
AWS Glue		:mod:`airflow.providers.amazon.aws.hooks.glue`	:mod:`airflow.providers.amazon.aws.operators.glue`	:mod:`airflow.providers.amazon.aws.sensors.glue`
AWS Lambda		:mod:`airflow.providers.amazon.aws.hooks.lambda_function`
Amazon Athena		:mod:`airflow.providers.amazon.aws.hooks.athena`	:mod:`airflow.providers.amazon.aws.operators.athena`	:mod:`airflow.providers.amazon.aws.sensors.athena`
Amazon CloudFormation		:mod:`airflow.providers.amazon.aws.hooks.cloud_formation`	:mod:`airflow.providers.amazon.aws.operators.cloud_formation`	:mod:`airflow.providers.amazon.aws.sensors.cloud_formation`
Amazon CloudWatch Logs		:mod:`airflow.providers.amazon.aws.hooks.logs`
Amazon DynamoDB		:mod:`airflow.providers.amazon.aws.hooks.aws_dynamodb`
Amazon EC2		:mod:`airflow.providers.amazon.aws.hooks.ec2`	:mod:`airflow.providers.amazon.aws.operators.ec2_start_instance`, :mod:`airflow.providers.amazon.aws.operators.ec2_stop_instance`,	:mod:`airflow.providers.amazon.aws.sensors.ec2_instance_state`
Amazon ECS	:doc:`How to use <howto/operator/amazon/aws/ecs>`		:mod:`airflow.providers.amazon.aws.operators.ecs`
Amazon EMR	:doc:`How to use <howto/operator/amazon/aws/emr>`	:mod:`airflow.providers.amazon.aws.hooks.emr`	:mod:`airflow.providers.amazon.aws.operators.emr_add_steps`, :mod:`airflow.providers.amazon.aws.operators.emr_create_job_flow`, :mod:`airflow.providers.amazon.aws.operators.emr_terminate_job_flow`, :mod:`airflow.providers.amazon.aws.operators.emr_modify_cluster`	:mod:`airflow.providers.amazon.aws.sensors.emr_base`, :mod:`airflow.providers.amazon.aws.sensors.emr_job_flow`, :mod:`airflow.providers.amazon.aws.sensors.emr_step`
Amazon Kinesis Data Firehose		:mod:`airflow.providers.amazon.aws.hooks.kinesis`
Amazon Redshift		:mod:`airflow.providers.amazon.aws.hooks.redshift`		:mod:`airflow.providers.amazon.aws.sensors.redshift`
Amazon SageMaker		:mod:`airflow.providers.amazon.aws.hooks.sagemaker`	:mod:`airflow.providers.amazon.aws.operators.sagemaker_base`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_endpoint_config`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_endpoint`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_model`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_training`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_transform`, :mod:`airflow.providers.amazon.aws.operators.sagemaker_tuning`	:mod:`airflow.providers.amazon.aws.sensors.sagemaker_base`, :mod:`airflow.providers.amazon.aws.sensors.sagemaker_endpoint`, :mod:`airflow.providers.amazon.aws.sensors.sagemaker_training`, :mod:`airflow.providers.amazon.aws.sensors.sagemaker_transform`, :mod:`airflow.providers.amazon.aws.sensors.sagemaker_tuning`
Amazon Simple Notification Service (SNS)		:mod:`airflow.providers.amazon.aws.hooks.sns`	:mod:`airflow.providers.amazon.aws.operators.sns`
Amazon Simple Queue Service (SQS)		:mod:`airflow.providers.amazon.aws.hooks.sqs`	:mod:`airflow.providers.amazon.aws.operators.sqs`	:mod:`airflow.providers.amazon.aws.sensors.sqs`
Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.hooks.s3`	:mod:`airflow.providers.amazon.aws.operators.s3_bucket`, :mod:`airflow.providers.amazon.aws.operators.s3_file_transform`, :mod:`airflow.providers.amazon.aws.operators.s3_copy_object`, :mod:`airflow.providers.amazon.aws.operators.s3_delete_objects`, :mod:`airflow.providers.amazon.aws.operators.s3_list`	:mod:`airflow.providers.amazon.aws.sensors.s3_key`, :mod:`airflow.providers.amazon.aws.sensors.s3_prefix`

Transfer operators and hooks

These integrations allow you to copy data from/to Amazon Web Services.

Source	Destination	Guide	Operator
All GCP services :ref:`[1] <integration:GCP-Discovery>`	Amazon Simple Storage Service (S3)	:doc:`How to use <howto/operator/amazon/aws/google_api_to_s3_transfer>`	:mod:`airflow.providers.amazon.aws.operators.google_api_to_s3_transfer`
Amazon DataSync	Amazon Simple Storage Service (S3)	:doc:`How to use <howto/operator/amazon/aws/datasync>`	:mod:`airflow.providers.amazon.aws.operators.datasync`
Amazon DynamoDB	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.dynamodb_to_s3`
Amazon Redshift	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.redshift_to_s3`
Amazon Simple Storage Service (S3)	Amazon Redshift	:doc:`How to use <howto/operator/amazon/aws/s3_to_redshift>`	:mod:`airflow.providers.amazon.aws.operators.s3_to_redshift`
Amazon Simple Storage Service (S3)	Snowflake		:mod:`airflow.providers.snowflake.operators.s3_to_snowflake`
Amazon Simple Storage Service (S3)	Apache Hive		:mod:`airflow.providers.apache.hive.operators.s3_to_hive`
Amazon Simple Storage Service (S3)	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/cloud_storage_transfer_service>`	:mod:`airflow.providers.google.cloud.operators.s3_to_gcs`, :mod:`airflow.providers.google.cloud.operators.cloud_storage_transfer_service`
Amazon Simple Storage Service (S3)	SSH File Transfer Protocol (SFTP)		:mod:`airflow.providers.amazon.aws.operators.s3_to_sftp`
Apache Hive	Amazon DynamoDB		:mod:`airflow.providers.amazon.aws.operators.hive_to_dynamodb`
Google Cloud Storage (GCS)	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.gcs_to_s3`
Internet Message Access Protocol (IMAP)	Amazon Simple Storage Service (S3)	:doc:`How to use <howto/operator/amazon/aws/imap_attachment_to_s3>`	:mod:`airflow.providers.amazon.aws.operators.imap_attachment_to_s3`
MongoDB	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.mongo_to_s3`
SSH File Transfer Protocol (SFTP)	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.sftp_to_s3`

:ref:`[1] <integration:AWS-Discovery-ref>` Those discovery-based operators use :class:`~airflow.providers.google.common.hooks.discovery_api.GoogleDiscoveryApiHook` to communicate with Google Services via the Google API Python Client. Please note that this library is in maintenance mode hence it won't fully support GCP in the future. Therefore it is recommended that you use the custom GCP Service Operators for working with the Google Cloud Platform.

Google

Airflow has support for the Google service.

All hooks are based on :class:`airflow.providers.google.common.hooks.base_google.GoogleBaseHook`. Some integration also use :mod:`airflow.providers.google.common.hooks.discovery_api`.

See the :doc:`GCP connection type <howto/connection/gcp>` documentation to configure connections to Google services.

GCP: Google Cloud Platform

Airflow has extensive support for the Google Cloud Platform.

Note

You can learn how to use Google Cloud Platform integrations by analyzing the source code of the Google Cloud Platform example DAGs

Service operators and hooks

These integrations allow you to perform various operations within the Google Cloud Platform.

Service name	Guide	Hook	Operator	Sensor
AutoML	:doc:`How to use <howto/operator/gcp/automl>`	:mod:`airflow.providers.google.cloud.hooks.automl`	:mod:`airflow.providers.google.cloud.operators.automl`
BigQuery	:doc:`How to use <howto/operator/gcp/bigquery>`	:mod:`airflow.providers.google.cloud.hooks.bigquery`	:mod:`airflow.providers.google.cloud.operators.bigquery`	:mod:`airflow.providers.google.cloud.sensors.bigquery`
BigQuery Data Transfer Service	:doc:`How to use <howto/operator/gcp/bigquery_dts>`	:mod:`airflow.providers.google.cloud.hooks.bigquery_dts`	:mod:`airflow.providers.google.cloud.operators.bigquery_dts`	:mod:`airflow.providers.google.cloud.sensors.bigquery_dts`
Bigtable	:doc:`How to use <howto/operator/gcp/bigtable>`	:mod:`airflow.providers.google.cloud.hooks.bigtable`	:mod:`airflow.providers.google.cloud.operators.bigtable`	:mod:`airflow.providers.google.cloud.sensors.bigtable`
Cloud Build	:doc:`How to use <howto/operator/gcp/cloud_build>`	:mod:`airflow.providers.google.cloud.hooks.cloud_build`	:mod:`airflow.providers.google.cloud.operators.cloud_build`
Compute Engine	:doc:`How to use <howto/operator/gcp/compute>`	:mod:`airflow.providers.google.cloud.hooks.compute`	:mod:`airflow.providers.google.cloud.operators.compute`
Cloud Data Loss Prevention (DLP)		:mod:`airflow.providers.google.cloud.hooks.dlp`	:mod:`airflow.providers.google.cloud.operators.dlp`
DataFusion	:doc:`How to use <howto/operator/gcp/datafusion>`	:mod:`airflow.providers.google.cloud.hooks.datafusion`	:mod:`airflow.providers.google.cloud.operators.datafusion`
Datacatalog	:doc:`How to use <howto/operator/gcp/datacatalog>`	:mod:`airflow.providers.google.cloud.hooks.datacatalog`	:mod:`airflow.providers.google.cloud.operators.datacatalog`
Dataflow		:mod:`airflow.providers.google.cloud.hooks.dataflow`	:mod:`airflow.providers.google.cloud.operators.dataflow`
Dataproc	:doc:`How to use <howto/operator/gcp/dataproc>`	:mod:`airflow.providers.google.cloud.hooks.dataproc`	:mod:`airflow.providers.google.cloud.operators.dataproc`
Datastore		:mod:`airflow.providers.google.cloud.hooks.datastore`	:mod:`airflow.providers.google.cloud.operators.datastore`
Cloud Functions	:doc:`How to use <howto/operator/gcp/functions>`	:mod:`airflow.providers.google.cloud.hooks.functions`	:mod:`airflow.providers.google.cloud.operators.functions`
Cloud Firestore	:doc:`How to use <howto/operator/gcp/firestore>`	:mod:`airflow.providers.google.firebase.hooks.firestore`	:mod:`airflow.providers.google.firebase.operators.firestore`
Cloud Key Management Service (KMS)		:mod:`airflow.providers.google.cloud.hooks.kms`
Cloud Life Sciences	:doc:`How to use <howto/operator/gcp/life_sciences>`	:mod:`airflow.providers.google.cloud.hooks.life_sciences`	:mod:`airflow.providers.google.cloud.operators.life_sciences`
Kubernetes Engine	:doc:`How to use <howto/operator/gcp/kubernetes_engine>`	:mod:`airflow.providers.google.cloud.hooks.kubernetes_engine`	:mod:`airflow.providers.google.cloud.operators.kubernetes_engine`
Machine Learning Engine		:mod:`airflow.providers.google.cloud.hooks.mlengine`	:mod:`airflow.providers.google.cloud.operators.mlengine`
Cloud Memorystore	:doc:`How to use <howto/operator/gcp/cloud_memorystore>`	:mod:`airflow.providers.google.cloud.hooks.cloud_memorystore`	:mod:`airflow.providers.google.cloud.operators.cloud_memorystore`
Natural Language	:doc:`How to use <howto/operator/gcp/natural_language>`	:mod:`airflow.providers.google.cloud.hooks.natural_language`	:mod:`airflow.providers.google.cloud.operators.natural_language`
Cloud Pub/Sub	:doc:`How to use <howto/operator/gcp/pubsub>`	:mod:`airflow.providers.google.cloud.hooks.pubsub`	:mod:`airflow.providers.google.cloud.operators.pubsub`	:mod:`airflow.providers.google.cloud.sensors.pubsub`
Cloud Spanner	:doc:`How to use <howto/operator/gcp/spanner>`	:mod:`airflow.providers.google.cloud.hooks.spanner`	:mod:`airflow.providers.google.cloud.operators.spanner`
Cloud Speech-to-Text	:doc:`How to use <howto/operator/gcp/speech_to_text>`	:mod:`airflow.providers.google.cloud.hooks.speech_to_text`	:mod:`airflow.providers.google.cloud.operators.speech_to_text`
Cloud SQL	:doc:`How to use <howto/operator/gcp/cloud_sql>`	:mod:`airflow.providers.google.cloud.hooks.cloud_sql`	:mod:`airflow.providers.google.cloud.operators.cloud_sql`
Cloud Stackdriver	:doc:`How to use <howto/operator/gcp/stackdriver>`	:mod:`airflow.providers.google.cloud.hooks.stackdriver`	:mod:`airflow.providers.google.cloud.operators.stackdriver`
Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/gcs>`	:mod:`airflow.providers.google.cloud.hooks.gcs`	:mod:`airflow.providers.google.cloud.operators.gcs`	:mod:`airflow.providers.google.cloud.sensors.gcs`
Storage Transfer Service	:doc:`How to use <howto/operator/gcp/cloud_storage_transfer_service>`	:mod:`airflow.providers.google.cloud.hooks.cloud_storage_transfer_service`	:mod:`airflow.providers.google.cloud.operators.cloud_storage_transfer_service`	:mod:`airflow.providers.google.cloud.sensors.cloud_storage_transfer_service`
Cloud Tasks		:mod:`airflow.providers.google.cloud.hooks.tasks`	:mod:`airflow.providers.google.cloud.operators.tasks`
Cloud Text-to-Speech	:doc:`How to use <howto/operator/gcp/text_to_speech>`	:mod:`airflow.providers.google.cloud.hooks.text_to_speech`	:mod:`airflow.providers.google.cloud.operators.text_to_speech`
Cloud Translation	:doc:`How to use <howto/operator/gcp/translate>`	:mod:`airflow.providers.google.cloud.hooks.translate`	:mod:`airflow.providers.google.cloud.operators.translate`
Cloud Video Intelligence	:doc:`How to use <howto/operator/gcp/video_intelligence>`	:mod:`airflow.providers.google.cloud.hooks.video_intelligence`	:mod:`airflow.providers.google.cloud.operators.video_intelligence`
Cloud Vision	:doc:`How to use <howto/operator/gcp/vision>`	:mod:`airflow.providers.google.cloud.hooks.vision`	:mod:`airflow.providers.google.cloud.operators.vision`

Transfer operators and hooks

These integrations allow you to copy data from/to Google Cloud Platform.

Source	Destination	Guide	Operator
All services :ref:`[1] <integration:GCP-Discovery>`	Amazon Simple Storage Service (S3)	:doc:`How to use <howto/operator/amazon/aws/google_api_to_s3_transfer>`	:mod:`airflow.providers.amazon.aws.operators.google_api_to_s3_transfer`
Amazon Simple Storage Service (S3)	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/cloud_storage_transfer_service>`	:mod:`airflow.providers.google.cloud.operators.s3_to_gcs`, :mod:`airflow.providers.google.cloud.operators.cloud_storage_transfer_service`
Apache Cassandra	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.cassandra_to_gcs`
Azure Data Lake Storage	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.adls_to_gcs`
Facebook Ads	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/facebook_ads_to_gcs>`	:mod:`airflow.providers.google.cloud.operators.facebook_ads_to_gcs`
Google BigQuery	MySQL		:mod:`airflow.providers.google.cloud.operators.bigquery_to_mysql`
Google BigQuery	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.bigquery_to_gcs`
Google BigQuery	Google BigQuery		:mod:`airflow.providers.google.cloud.operators.bigquery_to_bigquery`
Cloud Firestore	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/firestore>`	:mod:`airflow.providers.google.firebase.operators.firestore`
Google Cloud Storage (GCS)	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.gcs_to_s3`
Google Cloud Storage (GCS)	Google BigQuery		:mod:`airflow.providers.google.cloud.operators.gcs_to_bigquery`
Google Cloud Storage (GCS)	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/gcs_to_gcs>`, :doc:`How to use <howto/operator/gcp/cloud_storage_transfer_service>`	:mod:`airflow.providers.google.cloud.operators.gcs_to_gcs`, :mod:`airflow.providers.google.cloud.operators.cloud_storage_transfer_service`
Google Cloud Storage (GCS)	Google Drive		:mod:`airflow.providers.google.suite.operators.gcs_to_gdrive`
Google Cloud Storage (GCS)	SFTP	:doc:`How to use <howto/operator/gcp/gcs_to_sftp>`	:mod:`airflow.providers.google.cloud.operators.gcs_to_sftp`
Local	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/local_to_gcs>`	:mod:`airflow.providers.google.cloud.operators.local_to_gcs`
Microsoft SQL Server (MSSQL)	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.mssql_to_gcs`
MySQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.mysql_to_gcs`
PostgresSQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.postgres_to_gcs`
Presto	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/presto_to_gcs>`	:mod:`airflow.providers.google.cloud.operators.presto_to_gcs`
SFTP	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/sftp_to_gcs>`	:mod:`airflow.providers.google.cloud.operators.sftp_to_gcs`
SQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.sql_to_gcs`
Google Spreadsheet	Google Cloud Storage (GCS)	:doc:`How to use <howto/operator/gcp/sheets_to_gcs>`	:mod:`airflow.providers.google.cloud.operators.sheets_to_gcs`
Google Cloud Storage (GCS)	Google Spreadsheet	:doc:`How to use <howto/operator/gcp/gcs_to_sheets>`	:mod:`airflow.providers.google.suite.operators.gcs_to_sheets`

:ref:`[1] <integration:GCP-Discovery-ref>` Those discovery-based operators use :class:`~airflow.providers.google.common.hooks.discovery_api.GoogleDiscoveryApiHook` to communicate with Google Services via the Google API Python Client. Please note that this library is in maintenance mode hence it won't fully support Google in the future. Therefore it is recommended that you use the custom Google Service Operators for working with the Google services.

Other operators and hooks

Guide	Operator	Hook
:doc:`How to use <howto/operator/gcp/translate_speech>`	:mod:`airflow.providers.google.cloud.operators.translate_speech`

Google Marketing Platform

Note

You can learn how to use Google Marketing Platform integrations by analyzing the source code of the example DAGs.

Source	Destination	Guide	Operator	Sensor
Analytics360	:doc:`How to use <howto/operator/gcp/analytics>`	:mod:`airflow.providers.google.marketing_platform.hooks.analytics`	:mod:`airflow.providers.google.marketing_platform.operators.analytics`
Google Campaign Manager	:doc:`How to use <howto/operator/gcp/campaign_manager>`	:mod:`airflow.providers.google.marketing_platform.hooks.campaign_manager`	:mod:`airflow.providers.google.marketing_platform.operators.campaign_manager`	:mod:`airflow.providers.google.marketing_platform.sensors.campaign_manager`
Google Display&Video 360	:doc:`How to use <howto/operator/gcp/display_video>`	:mod:`airflow.providers.google.marketing_platform.hooks.display_video`	:mod:`airflow.providers.google.marketing_platform.operators.display_video`	:mod:`airflow.providers.google.marketing_platform.sensors.display_video`
Google Search Ads 360	:doc:`How to use <howto/operator/gcp/search_ads>`	:mod:`airflow.providers.google.marketing_platform.hooks.search_ads`	:mod:`airflow.providers.google.marketing_platform.operators.search_ads`	:mod:`airflow.providers.google.marketing_platform.sensors.search_ads`

Other Google operators and hooks

Service name	Guide	Hook	Operator
Google Ads	:doc:`How to use <howto/operator/gcp/ads>`	:mod:`airflow.providers.google.ads.hooks.ads`	:mod:`airflow.providers.google.ads.operators.ads`
Google Drive		:mod:`airflow.providers.google.suite.hooks.drive`
Cloud Firestore	:doc:`How to use <howto/operator/gcp/firestore>`	:mod:`airflow.providers.google.firebase.hooks.firestore`	:mod:`airflow.providers.google.firebase.operators.firestore`
Google Spreadsheet	:doc:`How to use <howto/operator/gcp/sheets>`	:mod:`airflow.providers.google.suite.hooks.sheets`	:mod:`airflow.providers.google.suite.operators.sheets`

Yandex.Cloud

Airflow has a limited support for the Yandex.Cloud.

See the :doc:`Yandex.Cloud connection type <howto/connection/yandexcloud>` documentation to configure connections to Yandex.Cloud.

All hooks are based on :class:`airflow.providers.yandex.hooks.yandex.YandexCloudBaseHook`.

Note

You can learn how to use Yandex.Cloud integrations by analyzing the example DAG

Service operators and hooks

These integrations allow you to perform various operations within the Yandex.Cloud.

Service name	Guide	Hook	Operator	Sensor
Base Classes	:doc:`How to use <howto/operator/yandexcloud>`	:mod:`airflow.providers.yandex.hooks.yandex`
Data Proc	:doc:`How to use <howto/operator/yandexcloud>`	:mod:`airflow.providers.yandex.hooks.yandexcloud_dataproc`	:mod:`airflow.providers.yandex.operators.yandexcloud_dataproc`

Service integrations

Service operators and hooks

These integrations allow you to perform various operations within various services.

Service name	Guide	Hook	Operator	Sensor
Atlassian Jira		:mod:`airflow.providers.jira.hooks.jira`	:mod:`airflow.providers.jira.operators.jira`	:mod:`airflow.providers.jira.sensors.jira`
Databricks		:mod:`airflow.providers.databricks.hooks.databricks`	:mod:`airflow.providers.databricks.operators.databricks`
Datadog		:mod:`airflow.providers.datadog.hooks.datadog`		:mod:`airflow.providers.datadog.sensors.datadog`
Pagerduty		:mod:`airflow.providers.pagerduty.hooks.pagerduty`
Dingding	:doc:`How to use <howto/operator/dingding>`	:mod:`airflow.providers.dingding.hooks.dingding`	:mod:`airflow.providers.dingding.operators.dingding`
Discord		:mod:`airflow.providers.discord.hooks.discord_webhook`	:mod:`airflow.providers.discord.operators.discord_webhook`
Facebook Ads		:mod:`airflow.providers.facebook.ads.hooks.ads`
IBM Cloudant		:mod:`airflow.providers.cloudant.hooks.cloudant`
Jenkins		:mod:`airflow.providers.jenkins.hooks.jenkins`	:mod:`airflow.providers.jenkins.operators.jenkins_job_trigger`
Opsgenie		:mod:`airflow.providers.opsgenie.hooks.opsgenie_alert`	:mod:`airflow.providers.opsgenie.operators.opsgenie_alert`
Qubole		:mod:`airflow.providers.qubole.hooks.qubole`, :mod:`airflow.providers.qubole.hooks.qubole_check`	:mod:`airflow.providers.qubole.operators.qubole`, :mod:`airflow.providers.qubole.operators.qubole_check`	:mod:`airflow.providers.qubole.sensors.qubole`
Salesforce		:mod:`airflow.providers.salesforce.hooks.salesforce`, :mod:`airflow.providers.salesforce.hooks.tableau`	:mod:`airflow.providers.salesforce.operators.tableau_refresh_workbook`	:mod:`airflow.providers.salesforce.sensors.tableau_job_status`
Segment		:mod:`airflow.providers.segment.hooks.segment`	:mod:`airflow.providers.segment.operators.segment_track_event`
Slack		:mod:`airflow.providers.slack.hooks.slack`, :mod:`airflow.providers.slack.hooks.slack_webhook`	:mod:`airflow.providers.slack.operators.slack`, :mod:`airflow.providers.slack.operators.slack_webhook`
Snowflake		:mod:`airflow.providers.snowflake.hooks.snowflake`	:mod:`airflow.providers.snowflake.operators.snowflake`, :mod:`airflow.providers.snowflake.operators.snowflake_to_slack`
Vertica		:mod:`airflow.providers.vertica.hooks.vertica`	:mod:`airflow.providers.vertica.operators.vertica`
Zendesk		:mod:`airflow.providers.zendesk.hooks.zendesk`

Transfer operators and hooks

These integrations allow you to perform various operations within various services.

Source	Destination	Guide	Operator
Google Cloud Storage (GCS)	Google Drive	:doc:`How to use <howto/operator/gcp/gcs_to_gdrive>`	:mod:`airflow.providers.google.suite.operators.gcs_to_gdrive`
Vertica	Apache Hive		:mod:`airflow.providers.apache.hive.operators.vertica_to_hive`
Vertica	MySQL		:mod:`airflow.providers.mysql.operators.vertica_to_mysql`

Software integrations

Software operators and hooks

These integrations allow you to perform various operations using various software.

Service name	Guide	Hook	Operator	Sensor
Celery				:mod:`airflow.providers.celery.sensors.celery_queue`
Docker		:mod:`airflow.providers.docker.hooks.docker`	:mod:`airflow.providers.docker.operators.docker`, :mod:`airflow.providers.docker.operators.docker_swarm`
Elasticsearch		:mod:`airflow.providers.elasticsearch.hooks.elasticsearch`
Exasol		:mod:`airflow.providers.exasol.hooks.exasol`	:mod:`airflow.providers.exasol.operators.exasol`
GNU Bash	:doc:`How to use <howto/operator/bash>`		:mod:`airflow.operators.bash`	:mod:`airflow.sensors.bash`
Kubernetes	:doc:`How to use <howto/operator/kubernetes>`	:mod:`airflow.providers.cncf.kubernetes.hooks.kubernetes`	:mod:`airflow.providers.cncf.kubernetes.operators.kubernetes_pod` :mod:`airflow.providers.cncf.kubernetes.operators.spark_kubernetes`	:mod:`airflow.providers.cncf.kubernetes.sensors.spark_kubernetes`
Microsoft SQL Server (MSSQL)		:mod:`airflow.providers.microsoft.mssql.hooks.mssql`, :mod:`airflow.providers.odbc.hooks.odbc`	:mod:`airflow.providers.microsoft.mssql.operators.mssql`
ODBC		:mod:`airflow.providers.odbc.hooks.odbc`
MongoDB		:mod:`airflow.providers.mongo.hooks.mongo`		:mod:`airflow.providers.mongo.sensors.mongo`
MySQL		:mod:`airflow.providers.mysql.hooks.mysql`	:mod:`airflow.providers.mysql.operators.mysql`
OpenFaaS		:mod:`airflow.providers.openfaas.hooks.openfaas`
Oracle		:mod:`airflow.providers.oracle.hooks.oracle`	:mod:`airflow.providers.oracle.operators.oracle`
Papermill	:doc:`How to use <howto/operator/papermill>`		:mod:`airflow.providers.papermill.operators.papermill`
PostgresSQL		:mod:`airflow.providers.postgres.hooks.postgres`	:mod:`airflow.providers.postgres.operators.postgres`
Presto		:mod:`airflow.providers.presto.hooks.presto`
Python		:doc:`How to use <howto/operator/python>`	:mod:`airflow.operators.python`	:mod:`airflow.sensors.python`
Redis		:mod:`airflow.providers.redis.hooks.redis`	:mod:`airflow.providers.redis.operators.redis_publish`	:mod:`airflow.providers.redis.sensors.redis_pub_sub`, :mod:`airflow.providers.redis.sensors.redis_key`
Samba		:mod:`airflow.providers.samba.hooks.samba`
Singularity			:mod:`airflow.providers.singularity.operators.singularity`
SQLite		:mod:`airflow.providers.sqlite.hooks.sqlite`	:mod:`airflow.providers.sqlite.operators.sqlite`

Transfer operators and hooks

These integrations allow you to copy data.

Source	Destination	Guide	Operator
Apache Hive	Samba		:mod:`airflow.providers.apache.hive.operators.hive_to_samba`
BigQuery	MySQL		:mod:`airflow.providers.google.cloud.operators.bigquery_to_mysql`
Microsoft SQL Server (MSSQL)	Apache Hive		:mod:`airflow.providers.apache.hive.operators.mssql_to_hive`
Microsoft SQL Server (MSSQL)	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.mssql_to_gcs`
MongoDB	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.mongo_to_s3`
MySQL	Apache Hive		:mod:`airflow.providers.apache.hive.operators.mysql_to_hive`
MySQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.mysql_to_gcs`
Oracle	Azure Data Lake Storage		:mod:`airflow.providers.microsoft.azure.operators.oracle_to_azure_data_lake_transfer`
Oracle	Oracle		:mod:`airflow.providers.oracle.operators.oracle_to_oracle_transfer`
PostgresSQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.postgres_to_gcs`
Presto	MySQL		:mod:`airflow.providers.mysql.operators.presto_to_mysql`
SQL	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.sql_to_gcs`
Vertica	Apache Hive		:mod:`airflow.providers.apache.hive.operators.vertica_to_hive`
Vertica	MySQL		:mod:`airflow.providers.mysql.operators.vertica_to_mysql`

Protocol integrations

Protocol operators and hooks

These integrations allow you to perform various operations within various services using standardized communication protocols or interface.

Service name	Guide	Hooks	Operator	Sensor
File Transfer Protocol (FTP)		:mod:`airflow.providers.ftp.hooks.ftp`		:mod:`airflow.providers.ftp.sensors.ftp`
Filesystem		:mod:`airflow.hooks.filesystem`		:mod:`airflow.sensors.filesystem`
Hypertext Transfer Protocol (HTTP)	:doc:`How to use <howto/operator/http/http>`	:mod:`airflow.providers.http.hooks.http`	:mod:`airflow.providers.http.operators.http`	:mod:`airflow.providers.http.sensors.http`
Internet Message Access Protocol (IMAP)		:mod:`airflow.providers.imap.hooks.imap`		:mod:`airflow.providers.imap.sensors.imap_attachment`
Java Database Connectivity (JDBC)		:mod:`airflow.providers.jdbc.hooks.jdbc`	:mod:`airflow.providers.jdbc.operators.jdbc`
SSH File Transfer Protocol (SFTP)		:mod:`airflow.providers.sftp.hooks.sftp`	:mod:`airflow.providers.sftp.operators.sftp`	:mod:`airflow.providers.sftp.sensors.sftp`
Secure Shell (SSH)		:mod:`airflow.providers.ssh.hooks.ssh`	:mod:`airflow.providers.ssh.operators.ssh`
Simple Mail Transfer Protocol (SMTP)			:mod:`airflow.providers.email.operators.email`
Windows Remote Management (WinRM)		:mod:`airflow.providers.microsoft.winrm.hooks.winrm`	:mod:`airflow.providers.microsoft.winrm.operators.winrm`
gRPC		:mod:`airflow.providers.grpc.hooks.grpc`	:mod:`airflow.providers.grpc.operators.grpc`

Transfer operators and hooks

These integrations allow you to copy data.

Source	Destination	Guide	Operator
Amazon Simple Storage Service (S3)	SSH File Transfer Protocol (SFTP)		:mod:`airflow.providers.amazon.aws.operators.s3_to_sftp`
Filesystem	Azure Blob Storage		:mod:`airflow.providers.microsoft.azure.operators.file_to_wasb`
Filesystem	Google Cloud Storage (GCS)		:mod:`airflow.providers.google.cloud.operators.local_to_gcs`
Internet Message Access Protocol (IMAP)	Amazon Simple Storage Service (S3)	:doc:`How to use <howto/operator/amazon/aws/imap_attachment_to_s3>`	:mod:`airflow.providers.amazon.aws.operators.imap_attachment_to_s3`
SSH File Transfer Protocol (SFTP)	Amazon Simple Storage Service (S3)		:mod:`airflow.providers.amazon.aws.operators.sftp_to_s3`