user
09/20/2021, 4:04 AMpython main.py check --config secrets/config.json
In my spec.json I have
"page-size": {
"type": "number",
"description": "Results per page"
},
Changing it to string
works, but I would expect number
to work.Marwan
01/17/2022, 9:59 AMAriyo Kabir
01/26/2022, 5:38 PMaccounts
stream to stripe. I am having trouble putting the schema together. I used the openAPIspec2Json
script but it creates the schema with references, and when I try to use the --stand-alone
flag, I get circular dependency error
. Is there any work around for this or do I have to stitch the schema together manually?Jags (Kloud.io)
01/27/2022, 11:42 AMTimed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@2575f671. Client view of cluster state is {type=REPLICA_SET, servers=[{address=<http://dds-pub.mongodb.ap-southeast-5.rds.aliyuncs.com:3717|dds-pub.mongodb.ap-southeast-5.rds.aliyuncs.com:3717>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}
Can you help me how to fix this?Faris Alfa Mauludy
01/27/2022, 1:45 PMAli Mojiz
01/27/2022, 3:30 PMDaniel Eduardo Portugal Revilla
01/27/2022, 5:13 PMDaniel Eduardo Portugal Revilla
01/27/2022, 5:17 PMJove Zhong
01/27/2022, 7:01 PMDaniel Eduardo Portugal Revilla
01/27/2022, 7:46 PMTomas Balciunas
01/28/2022, 5:09 AM2022-01-28 04:55:07 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed.
errors: $: string found, object expected
2022-01-28 04:55:07 ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: "type"
Can you please give me any insight into this issue. I can confirm that the data points are as expected by running the script locally.will
01/28/2022, 11:33 AM"supported_sync_modes": [
"full_refresh",
"incremental"
],
"source_defined_cursor": True,
"default_cursor_field":["json_data","properties","updated_at"],
"source_defined_primary_key": ["json_data","properties","_id"],
Am I missing something?Kévin Maschtaler
01/28/2022, 3:16 PMKévin Maschtaler
01/28/2022, 4:37 PMPhoebe Yang
01/28/2022, 4:40 PMlass ServicesnowApi(HttpStream):
url_base = "https://.com/api/now/v1/"
# Set this as a noop.
primary_key = None
def __init__(self, limit: str, sys_created_from: str, sys_created_to: str, **kwargs):
super().__init__(**kwargs)
# Here's where we set the variable from our input to pass it down to the source.
self.limit = limit
self.sys_created_from = sys_created_from
self.sys_created_to = sys_created_to
def path(self, **kwargs) -> str:
# This defines the path to the endpoint that we want to hit.
limit = self.limit
sys_created_from = self.sys_created_from
sys_created_to = self.sys_created_to
return f"table/incident?sysparm_offset=0&sysparm_limit={limit}&sysparm_query=sys_created_on>={sys_created_from} 08:00^sys_created_on<{sys_created_to} 08:00^active=ISNOTEMPTY"
def request_params(
self,
stream_state: Mapping[str, Any],
stream_slice: Mapping[str, Any] = None,
next_page_token: Mapping[str, Any] = None,
) -> MutableMapping[str, Any]:
# The api requires that we include the Pokemon name as a query param so we do that in this method.
limit = self.limit
sys_created_from = self.sys_created_from
sys_created_to = self.sys_created_to
return {"limit": limit, "sys_created_from":sys_created_from, "sys_created_to":sys_created_to}
def parse_response(
self,
response: requests.Response,
stream_state: Mapping[str, Any],
stream_slice: Mapping[str, Any] = None,
next_page_token: Mapping[str, Any] = None,
) -> Iterable[Mapping]:
# The response is a simple JSON whose schema matches our stream's schema exactly,
# so we just return a list containing the response.
return [response.json()]
def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
# While the PokeAPI does offer pagination, we will only ever retrieve one Pokemon with this implementation,
# so we just return None to indicate that there will never be any more pages in the response.
return None
Siddharth Putuvely
01/29/2022, 12:33 PM2022-01-29 11:54:15 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed.
errors: $.record.data: string found, object expected
2022-01-29 11:54:15 ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: {"type":"RECORD","record":{"stream":"email","data":"{\n \"id\": \"8994962\"\n}","emitted_at":1643457255165}}
flow
01/30/2022, 9:09 PMKerim Tricic
01/31/2022, 12:11 AMJordan Velich
01/31/2022, 12:22 AMuser
01/31/2022, 7:51 PMMartin Prejean
02/01/2022, 5:51 PM<https://mycompany.service-now.com/api/now/v1/table/incident?sysparm_offset=0&sysparm_limit=1000000&sysparm_query=sys_created_on>>=2020-01-01 08:00^sys_created_on<2021-12-02 08:00^active=ISNOTEMPTY
Alessandro Duico
02/02/2022, 10:48 AMAlessandro Duico
02/02/2022, 3:15 PMclass ServicesnowApi(HttpStream):
url_base = "<https://xxx.service-now.com/api/now/v1/>"
# Set this as a noop.
primary_key = None
# Save the state every 100 records
state_checkpoint_interval = 100
page_size = 100
cursor_field = "sys_updated_on"
def __init__(self, limit: str, sys_created_from: str, **kwargs):
super().__init__(**kwargs)
# Here's where we set the variable from our input to pass it down to the source.
self.limit = limit
self.sys_created_from = sys_created_from
def path(self, **kwargs) -> str:
# This defines the path to the endpoint that we want to hit.
limit = self.limit
sys_created_from = self.sys_created_from
return f"table/incident?sysparm_offset=0&sysparm_limit={limit}&sysparm_query=sys_created_on>={sys_created_from} 00:00^active=ISNOTEMPTY"
def request_params(
self,
stream_state: Mapping[str, Any],
stream_slice: Mapping[str, Any] = None,
next_page_token: Mapping[str, Any] = None,
) -> MutableMapping[str, Any]:
limit = self.limit
sys_created_from = self.sys_created_from
return {"limit": limit, "sys_created_from":sys_created_from}
def parse_response(
self,
response: requests.Response,
stream_state: Mapping[str, Any],
stream_slice: Mapping[str, Any] = None,
next_page_token: Mapping[str, Any] = None,
) -> Iterable[Mapping]:
result = response.json()['result']
return result
def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
return None
Daniel Eduardo Portugal Revilla
02/02/2022, 6:49 PMNathan Gille
02/02/2022, 10:15 PM{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "FAILED", "message": "Please use valid credentials json file. Error: Invalid control character at: line 1 column 171 (char 170)"}}
when I run the check
command. It doesn't seem to like the \n
s in the private key. Any advice?Nathan Gille
02/03/2022, 5:51 AMFamezilla Channel
02/03/2022, 1:10 PMKhristina Rustanovich
02/03/2022, 3:56 PMMaxime edfeed
02/06/2022, 11:37 PMtap | target
). Is this how Airbyte works?Jackson Clarke
02/07/2022, 9:15 AM