Skip to content

Commit b54a2de

Browse files
authored
Local filesystem to Google Drive Operator - system tests migration (AIP-47) (#26797)
1 parent b4cef6d commit b54a2de

File tree

6 files changed

+43
-8
lines changed

6 files changed

+43
-8
lines changed

‎docs/apache-airflow-providers-google/example-dags.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,12 @@
1717
1818
Example DAGs
1919
============
20-
2120
You can learn how to use Google integrations by analyzing the source code of the example DAGs:
2221

2322
* `Google Ads <https://github.com/apache/airflow/tree/providers-google/8.1.0/tests/system/providers/google/ads>`__
2423
* `Google Cloud (legacy) <https://github.com/apache/airflow/tree/providers-google/8.0.0/airflow/providers/google/cloud/example_dags>`__
2524
* `Google Cloud <https://github.com/apache/airflow/tree/providers-google/8.0.0/tests/system/providers/google>`__
2625
* `Google Firebase <https://github.com/apache/airflow/tree/providers-google/8.1.0/tests/system/providers/google/firebase>`__
2726
* `Google Marketing Platform <https://github.com/apache/airflow/tree/providers-google/8.0.0/airflow/providers/google/marketing_platform/example_dags>`__
28-
* `Google Workplace <https://github.com/apache/airflow/tree/providers-google/8.0.0/airflow/providers/google/suite/example_dags>`__ (formerly Google Suite)
27+
* `Google Workplace <https://github.com/apache/airflow/tree/providers-google/8.3.0/tests/system/providers/google/suite>`__ (formerly Google Suite)
2928
* `Google LevelDB <https://github.com/apache/airflow/tree/providers-google/8.0.0/tests/system/providers/google/leveldb>`__

‎docs/apache-airflow-providers-google/operators/transfer/local_to_drive.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,11 @@ LocalFilesystemToGoogleDriveOperator
3535
data from local filesystem to GoogleDrive.
3636

3737
When you use this operator, you can upload a list of files.
38+
39+
Below is an example of using this operator to upload data from local filesystem to Google Drive.
40+
41+
.. exampleinclude:: /../../tests/system/providers/google/suite/example_local_to_drive.py
42+
:language: python
43+
:dedent: 0
44+
:start-after: [START howto_operator_local_to_drive_upload_single_file]
45+
:end-before: [END howto_operator_local_to_drive_upload_single_file]

‎airflow/providers/google/suite/example_dags/example_local_to_drive.py renamed to ‎tests/system/providers/google/suite/example_local_to_drive.py

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@
1717
# under the License.
1818
"""
1919
Example DAG using LocalFilesystemToGoogleDriveOperator.
20+
21+
Using this operator requires the following additional scopes:
22+
https://www.googleapis.com/auth/drive
2023
"""
2124
from __future__ import annotations
2225

@@ -26,16 +29,27 @@
2629
from airflow import models
2730
from airflow.providers.google.suite.transfers.local_to_drive import LocalFilesystemToGoogleDriveOperator
2831

29-
SINGLE_FILE_LOCAL_PATHS = [Path("test1")]
30-
MULTIPLE_FILES_LOCAL_PATHS = [Path("test1"), Path("test2")]
31-
DRIVE_FOLDER = Path("test-folder")
32+
DAG_ID = "example_local_to_drive"
33+
34+
FILE_NAME_1 = "test1"
35+
FILE_NAME_2 = "test2"
36+
37+
LOCAL_PATH = str(Path(__file__).parent / "resources")
38+
39+
SINGLE_FILE_LOCAL_PATHS = [str(Path(LOCAL_PATH) / FILE_NAME_1)]
40+
MULTIPLE_FILES_LOCAL_PATHS = [str(Path(LOCAL_PATH) / FILE_NAME_1), str(Path(LOCAL_PATH) / FILE_NAME_2)]
41+
42+
DRIVE_FOLDER = "test-folder"
43+
3244

3345
with models.DAG(
34-
"example_local_to_drive",
46+
DAG_ID,
47+
schedule="@once",
3548
start_date=datetime(2021, 1, 1),
3649
catchup=False,
3750
tags=["example"],
3851
) as dag:
52+
3953
# [START howto_operator_local_to_drive_upload_single_file]
4054
upload_single_file = LocalFilesystemToGoogleDriveOperator(
4155
task_id="upload_single_file",
@@ -53,4 +67,19 @@
5367
)
5468
# [END howto_operator_local_to_drive_upload_multiple_files]
5569

56-
upload_single_file >> upload_multiple_files
70+
(
71+
# TEST BODY
72+
upload_single_file
73+
>> upload_multiple_files
74+
)
75+
76+
from tests.system.utils.watcher import watcher
77+
78+
# This test needs watcher in order to properly mark success/failure
79+
# when "tearDown" task with trigger rule is part of the DAG
80+
list(dag.tasks) >> watcher()
81+
82+
from tests.system.utils import get_test_run # noqa: E402
83+
84+
# Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
85+
test_run = get_test_run(dag)

‎airflow/providers/google/suite/example_dags/__init__.py renamed to ‎tests/system/providers/google/suite/resources/__init__.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
#
21
# Licensed to the Apache Software Foundation (ASF) under one
32
# or more contributor license agreements. See the NOTICE file
43
# distributed with this work for additional information

‎tests/system/providers/google/suite/resources/test1

Whitespace-only changes.

‎tests/system/providers/google/suite/resources/test2

Whitespace-only changes.

0 commit comments

Comments
 (0)