@here I have noticed that while transferring data ...
# office-hour
h
@here I have noticed that while transferring data from mongodb to elastic search i am facing an error like from _id(mongo) to _id(elastic search) error parsing. So what possible solution should be for that issue??? And also i think this issue will be for all NoSQL sources and destinations.
s
Can you share the full error? Sometimes this comes up because of reserved keywords being used in the wrong place
h
Yes you are correct, reserved keyword used, _id(mongo), and have to use for identification. Please find the below error.
Copy code
2023-05-15 10:24:40 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...
2023-05-15 10:24:40 INFO i.a.p.j.e.LoggingJobErrorReportingClient(reportJobFailureReason):26 - Report Job Error -> workspaceId: a8bf85f2-02b9-4fc5-ac04-29489cd8a5ca, dockerImage: airbyte/destination-elasticsearch:0.1.6, failureReason: io.airbyte.config.FailureReason@7abf87c3[failureOrigin=destination,failureType=system_error,internalMessage=java.lang.Exception: failed to write bulk records for index: user 
error type: mapper_parsing_exception
 reason: failed to parse field [_id] of type [_id] in document with id '7d05b908-31f3-4423-a813-1224bb33de5e'. Preview of field's value: '6461fd37a1d00941a0339c05',externalMessage=Something went wrong in the connector. See the logs for more details.,metadata=io.airbyte.config.Metadata@2d713f3c[additionalProperties={attemptNumber=2, jobId=150, connector_command=write, from_trace_message=true}],stacktrace=java.lang.Exception: failed to write bulk records for index: user 
error type: mapper_parsing_exception
 reason: failed to parse field [_id] of type [_id] in document with id '7d05b908-31f3-4423-a813-1224bb33de5e'. Preview of field's value: '6461fd37a1d00941a0339c05'
	at io.airbyte.integrations.destination.elasticsearch.ElasticsearchAirbyteMessageConsumerFactory.lambda$recordWriterFunction$3(ElasticsearchAirbyteMessageConsumerFactory.java:97)
	at io.airbyte.integrations.destination.record_buffer.InMemoryRecordBufferingStrategy.flushAll(InMemoryRecordBufferingStrategy.java:86)
	at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:172)
	at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:64)
	at io.airbyte.integrations.base.AirbyteMessageConsumer$1.close(AirbyteMessageConsumer.java:59)
	at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:147)
	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:97)
	at io.airbyte.integrations.destination.elasticsearch.ElasticsearchDestination.main(ElasticsearchDestination.java:31)
,retryable=<null>,timestamp=1684146280273,additionalProperties={}], metadata: {connector_repository=airbyte/destination-elasticsearch, workspace_url=<http://localhost:8000/workspaces/a8bf85f2-02b9-4fc5-ac04-29489cd8a5ca>, failure_type=system_error, connector_command=write, connector_name=ElasticSearch, workspace_id=a8bf85f2-02b9-4fc5-ac04-29489cd8a5ca, airbyte_version=0.44.0, connection_url=<http://localhost:8000/workspaces/a8bf85f2-02b9-4fc5-ac04-29489cd8a5ca/connections/545edd40-3039-4d2c-be59-657ffe810d9e>, connector_definition_id=68f351a7-2745-4bef-ad7f-996b8e51bb8c, failure_origin=destination, connection_id=545edd40-3039-4d2c-be59-657ffe810d9e, connector_release_stage=alpha, job_id=150, deployment_mode=OSS}
2023-05-15 10:24:40 INFO i.a.v.j.JsonSchemaValidator(test):121 - JSON schema validation failed.
@Sunny Hashmi pls can you look into it?