https://datahubproject.io logo
Join Slack
Powered by
Hi there, what would be the cause for ```source pr...
# troubleshoot
m
Hi there, what would be the cause for
Copy code
source produced an invalid metadata work unit: MetadataChangeEventClass
I'm trying to ingest some workbooks from tableau, and there's one particular one that keeps failing but i wasn't sure why.
o
It's hard to tell without the value in question. Is this all the logs give? Is it possible to run this with a debugger and inspect the failing event?
b
@modern-artist-55754 Were you able to solve it ? I am facing the same issue as well..
m
@broad-tomato-45373 we notice there was an issue with one of the workbook, that return more exceed 2000 nodes limit that causes the malformed result.
b
@modern-artist-55754 Thanks for the reply. didn't get the part 2000 nodes limit on which ?
m
@broad-tomato-45373 that’s the tableau GraphQL response limit, it won’t return more than 2000 nodes. https://community.tableau.com/s/question/0D54T00000YuK60SAF/metadata-query-nodelimitexceeded-error
b
@modern-artist-55754 okay thanks for the help. Will definitely look into this direction as well.
o
It seems like this is an issue with the Tableau side sending invalid responses when the node limit is reached, not an error from our end. What result are you expecting when this occurs? I don't think we can preemptively detect when this is going to occur since it's a Tableau side error.
m
What do you refer to when saying Tableau is sending invalid responses? This is the an example of a complete error I am getting and I do not see anything wrong.
Copy code
ValueError: source produced an invalid metadata work unit: MetadataChangeProposalWrapper(entityType='chart', changeType='UPSERT', entityUrn='urn:li:chart:(tableau,leboncoin.cac3b4a9-7c0f-204b-5262-da74ca73f570)', entityKeyAspect=None, auditHeader=None, aspectName='inputFields', aspect=InputFieldsClass({'fields': [InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Category)', 'schemaField': SchemaFieldClass({'fieldPath': 'Category', 'jsonPath': None, 'nullable': False, 'description': '', 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': StringTypeClass({})}), 'nativeDataType': 'STRING', 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})}), InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Nb Visits)', 'schemaField': SchemaFieldClass({'fieldPath': 'Nb Visits', 'jsonPath': None, 'nullable': False, 'description': '', 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': NumberTypeClass({})}), 'nativeDataType': 'INTEGER', 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})}), InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Parcours)', 'schemaField': SchemaFieldClass({'fieldPath': 'Parcours', 'jsonPath': None, 'nullable': False, 'description': '', 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': NullTypeClass({})}), 'nativeDataType': None, 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})}), InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Facture CB  - Visiteurs)', 'schemaField': SchemaFieldClass({'fieldPath': 'Volume Facture CB  - Visiteurs', 'jsonPath': None, 'nullable': False, 'description': "formula: sum (iif ([Step] = 'confirmation_cb',[nb_uv_id],0))", 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': NumberTypeClass({})}), 'nativeDataType': 'INTEGER', 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})}), InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Facture CB )', 'schemaField': SchemaFieldClass({'fieldPath': 'Volume Facture CB ', 'jsonPath': None, 'nullable': False, 'description': "formula: sum (iif ([Step] = 'confirmation_cb',[Nb Visits],0))", 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': NumberTypeClass({})}), 'nativeDataType': 'INTEGER', 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})}), InputFieldClass({'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Payment CB)', 'schemaField': SchemaFieldClass({'fieldPath': 'Volume Payment CB', 'jsonPath': None, 'nullable': False, 'description': "formula: sum (iif ([Step] = 'payment_cb',[Nb Visits],0))", 'label': None, 'created': None, 'lastModified': None, 'type': SchemaFieldDataTypeClass({'type': NumberTypeClass({})}), 'nativeDataType': 'INTEGER', 'recursive': False, 'globalTags': None, 'glossaryTerms': None, 'isPartOfKey': False, 'isPartitioningKey': None, 'jsonProps': None})})]}), systemMetadata=SystemMetadataClass({'lastObserved': 1673867352235, 'runId': 'tableau-2023_01_16-10_24_15', 'registryName': None, 'registryVersion': None, 'properties': None}))
Same error, formatted:
Copy code
MetadataChangeProposalWrapper(entityType = 'chart', changeType = 'UPSERT', entityUrn = 'urn:li:chart:(tableau,leboncoin.cac3b4a9-7c0f-204b-5262-da74ca73f570)', entityKeyAspect = None, auditHeader = None, aspectName = 'inputFields', aspect = InputFieldsClass({
    'fields': [InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Category)',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Category',
            'jsonPath': None,
            'nullable': False,
            'description': '',
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': StringTypeClass({})
            }),
            'nativeDataType': 'STRING',
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    }), InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Nb Visits)',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Nb Visits',
            'jsonPath': None,
            'nullable': False,
            'description': '',
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': NumberTypeClass({})
            }),
            'nativeDataType': 'INTEGER',
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    }), InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Parcours)',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Parcours',
            'jsonPath': None,
            'nullable': False,
            'description': '',
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': NullTypeClass({})
            }),
            'nativeDataType': None,
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    }), InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Facture CB  - Visiteurs)',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Volume Facture CB  - Visiteurs',
            'jsonPath': None,
            'nullable': False,
            'description': "formula: sum (iif ([Step] = 'confirmation_cb',[nb_uv_id],0))",
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': NumberTypeClass({})
            }),
            'nativeDataType': 'INTEGER',
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    }), InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Facture CB )',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Volume Facture CB ',
            'jsonPath': None,
            'nullable': False,
            'description': "formula: sum (iif ([Step] = 'confirmation_cb',[Nb Visits],0))",
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': NumberTypeClass({})
            }),
            'nativeDataType': 'INTEGER',
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    }), InputFieldClass({
        'schemaFieldUrn': 'urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:tableau,leboncoin.27d1404d-d0f5-4670-5b20-1f1b9626ce52,PROD),Volume Payment CB)',
        'schemaField': SchemaFieldClass({
            'fieldPath': 'Volume Payment CB',
            'jsonPath': None,
            'nullable': False,
            'description': "formula: sum (iif ([Step] = 'payment_cb',[Nb Visits],0))",
            'label': None,
            'created': None,
            'lastModified': None,
            'type': SchemaFieldDataTypeClass({
                'type': NumberTypeClass({})
            }),
            'nativeDataType': 'INTEGER',
            'recursive': False,
            'globalTags': None,
            'glossaryTerms': None,
            'isPartOfKey': False,
            'isPartitioningKey': None,
            'jsonProps': None
        })
    })]
}), systemMetadata = SystemMetadataClass({
    'lastObserved': 1673867352235,
    'runId': 'tableau-2023_01_16-10_24_15',
    'registryName': None,
    'registryVersion': None,
    'properties': None
}))
o
but only when the warning about NODE_LIMIT_EXCEEDED appears. If the same recipe is run adjusting the 
page_size
option, the warning stops appearing and the error also
This is what I mean, the error does not occur when Tableau sends the expected response, but when we try to map the response data for an error response (too many nodes) an error occurs, right? It seems like there exists both an understanding of the issue and a workaround on the Tableau GraphQL end to get a valid result, so I'm not sure what more you are expecting on our end. If you have specific requirements for error handling in this case it would make sense to set up a Github issue or feature request with those specific requirements, but we can't fix the underlying fact that the Tableau GraphQL API is rejecting the request and not returning the expected data to populate our model. Rather than pushing out partial incorrect data it is failing.
4 Views
Open in Slack
PreviousNext
Powered by