Philip Johnson
01/16/2023, 6:59 PMISO8601Error("Unable to parse duration string '1d'") , which is strange. Previously, in the tutorial this was the value shown as an example, but now it looks like it's P1D for both step and cursor granularity. If I try and use those locally, I get a different error: Something went wrong in the connector. See the logs for more details...line 190, in _parse_timedelta\n assert parts is not None\nAssertionError\n.
Can anyone provide a bit of guidance on what the step and cursor_granularity should actually be here?
Coming from the stream slicer section in the yaml file:
stream_slicer:
type: "DatetimeStreamSlicer"
start_datetime:
datetime: "{{ config['start_date'] }}"
datetime_format: "%Y-%m-%d"
end_datetime:
datetime: "{{ now_utc() }}"
datetime_format: "%Y-%m-%d %H:%M:%S.%f+00:00"
step: "1d"
datetime_format: "%Y-%m-%d"
cursor_field: "{{ options['stream_cursor_field'] }}"
cursor_granularity: "day"
$options:
name: "austin_permits"
primary_key: "permit_number"
path: "/3syk-w9eu.json?$where=issue_date>='{{config['start_date'] or 'latest'}}'"
cursor_field: "issue_date"Sebastian Brickel
01/23/2023, 8:21 AMPhilip Johnson
01/27/2023, 3:35 PMSebastian Brickel
01/27/2023, 3:39 PMPhilip Johnson
01/27/2023, 3:39 PMP1D : (Philip Johnson
01/27/2023, 3:39 PMSebastian Brickel
01/27/2023, 3:41 PMPhilip Johnson
01/27/2023, 3:42 PMSebastian Brickel
01/27/2023, 3:43 PM