这个问题发生在etl创建中，这是一个使用panda进行日期转换的问题。有人能帮帮我吗?
df_delivered = df.loc[(df['state'] == 2) | (df['state'] == 4) | (df['state'] == 7) | (df['direction'] == 1) | (df['delivered_at'] > last_update)]
df_delivered['Delivered'] = 1
df1 = df.merge(df_delivered, left_index=True, right_index=True, how='outer', suffixes=('', '_y'))
df1.drop(df1.filter(regex='_y$').columns, axis=1, inplace=True)
df = df1
和
上述异常是导致以下异常的直接原因:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py", line 1008, in _cmp_method
other = self._validate_comparison_value(other)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py", line 543, in _validate_comparison_value
raise InvalidComparison(other) from err
pandas.core.arrays.datetimelike.InvalidComparison: 2019-01-01 00:00:00
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1509, in _execute_task
result = execute_callable(context=context)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py", line 151, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py", line 162, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/etl/dw_reports/etl_communication_customer_engagement.py", line 62, in etl_communication_customer_engagement
df_delivered = df.loc[(df['state'] == 2) | (df['state'] == 4) | (df['state'] == 7) | (df['direction'] == 1) | (df['delivered_at'] > last_update)]
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/ops/common.py", line 69, in new_method
return method(self, other)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/arraylike.py", line 48, in __gt__
return self._cmp_method(other, operator.gt)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/series.py", line 5502, in _cmp_method
res_values = ops.comparison_op(lvalues, rvalues, op)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/ops/array_ops.py", line 270, in comparison_op
res_values = op(lvalues, rvalues)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/ops/common.py", line 69, in new_method
return method(self, other)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/arraylike.py", line 48, in __gt__
return self._cmp_method(other, operator.gt)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py", line 1010, in _cmp_method
return invalid_comparison(self, other, op)
File "/home/airflow/.local/lib/python3.9/site-packages/pandas/core/ops/invalid.py", line 34, in invalid_comparison
raise TypeError(f"Invalid comparison between dtype={left.dtype} and {typ}")
TypeError: Invalid comparison between dtype=datetime64[ns, UTC-03:00] and datetime
[2023-01-09, 13:54:11 UTC] {local_task_job.py:154} INFO - Task exited with return code 1
[2023-01-09, 13:54:11 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check`
似乎这个比较可能有问题:
(df("delivered_at")在last_update)
如果last_update在这里传递值，它需要与df['delivered_at']列中的数据具有相同/可比较的类型。
顺便说一句。这可以通过以下命令实现:
df。Loc [:， ('delivered_at')] = pandas.to_datetime(df。loc:, (delivered_at)])