https://datahubproject.io logo
Join Slack
Powered by
trying with the new React UI, getting exception wh...
# ui
c
trying with the new React UI, getting exception while doing searches,
Copy code
Exception while fetching data (/search) : java.lang.RuntimeException: Failed to execute search: entity type DATASET, query demand, filters: [], start: 0, count: 10
same errors when browsing these datasets. Doesn't seem to be data issues, as these are working fine with the Amber UI running side by side.
b
Hmmm you have all the containers up? Ember is working?
React served out of 9002?
c
yes, think so, Ember one is working fine
b
I'm not able to repro locally. You verified that sample data is in the index + DB, search working in Ember.. And you're pulling the latest react container using 
quickstart-react
?
Could be something opaque. Would be good to sync to see if there's some deep bug here
c
thanks for checking on it. yes, think I have pulled the latest. Seems like it may have something to do with certain datasets I ingested. React UI would blow up when I tried to access those datasets or search results contain them, while ember UI is fine.
Here is an example:
Copy code
{
  "snapshot": {
    "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,landing-db.data,DEV)",
    "aspects": [
      {
        "com.linkedin.common.Ownership": {
          "owners": [
            {
              "owner": "urn:li:corpuser:enterprisearchitect",
              "type": "DATAOWNER"
            }
          ],
          "lastModified": {
            "actor": "urn:li:corpuser:enterprisearchitect",
            "time": 1608306715000
          }
        }
      },
      {
        "com.linkedin.dataset.DatasetProperties": {
          "description": "Data output from listening to Manufacturing Order events (completed only) and determining the full BOM tree for the material requirements that went into that order.",
          "tags": [],
          "customProperties": {
            "CrawlerSchemaDeserializerVersion": "1.0",
            "CrawlerSchemaSerializerVersion": "1.0",
            "UPDATED_BY_CRAWLER": "requirements-firehose",
            "averageRecordSize": "9",
            "classification": "parquet",
            "compressionType": "none",
            "objectCount": "5532",
            "recordCount": "26505986",
            "sizeKey": "302519150",
            "typeOfData": "file"
          }
        }
      },
      {
        "com.linkedin.schema.SchemaMetadata": {
          "schemaName": "landing-db.data-schema",
          "platform": "urn:li:dataPlatform:hive",
          "version": 0,
          "created": {
            "actor": "urn:li:corpuser:enterprisearchitect",
            "time": 1608306715000
          },
          "lastModified": {
            "actor": "urn:li:corpuser:enterprisearchitect",
            "time": 1613693715000
          },
          "hash": "",
          "platformSchema": {
            "com.linkedin.schema.KafkaSchema": {
              "documentSchema": "{\"type\":\"record\",\"name\":\"landing-db.data-schema\",\"namespace\":\"com.linkedin.dataset\",\"doc\":\"Data output from listening to Manufacturing Order events (completed only) and determining the full BOM tree for the material requirements that went into that order.\",\"fields\":[{\"name\":\"manufacturingstartdate\",\"type\":[\"string\"]},{\"name\":\"component\",\"type\":[\"string\"]},{\"name\":\"componentquantity\",\"type\":[\"double\"]}]}"
            }
          },
          "fields": [
            {
              "fieldPath": "manufacturingstartdate",
              "description": "The date the manufacturing started. This is when the material would have needed to be available.",
              "jsonPath": "",
              "nullable": false,
              "recursive": false,
              "nativeDataType": "string",
              "type": {
                "type": {
                  "com.linkedin.schema.StringType": {}
                }
              }
            },
            {
              "fieldPath": "component",
              "description": "The part description (including any -.) of the required material.",
              "jsonPath": "",
              "nullable": false,
              "recursive": false,
              "nativeDataType": "string",
              "type": {
                "type": {
                  "com.linkedin.schema.StringType": {}
                }
              }
            },
            {
              "fieldPath": "componentquantity",
              "description": "How much of the required component was needed to manufacture the finished good (including scrap)",
              "jsonPath": "",
              "nullable": false,
              "recursive": false,
              "nativeDataType": "double",
              "type": {
                "type": {
                  "com.linkedin.schema.FixedType": {}
                }
              }
            }
          ]
        }
      },
      {
        "com.linkedin.dataset.UpstreamLineage": {
          "upstreams": [
            {
              "dataset": "urn:li:dataset:(urn:li:dataPlatform:mssql,Operations.BillOfMaterial_Base,PROD)",
              "type": "TRANSFORMED",
              "auditStamp": {
                "actor": "urn:li:corpuser:enterprisearchitect",
                "time": 1613762687145
              }
            },
            {
              "dataset": "urn:li:dataset:(urn:li:dataPlatform:mssql,Operations.Components_Base,PROD)",
              "type": "TRANSFORMED",
              "auditStamp": {
                "actor": "urn:li:corpuser:enterprisearchitect",
                "time": 1613762687145
              }
            },
            {
              "dataset": "urn:li:dataset:(urn:li:dataPlatform:mssql,dbo.INV_MATERIAL_BASE,PROD)",
              "type": "TRANSFORMED",
              "auditStamp": {
                "actor": "urn:li:corpuser:enterprisearchitect",
                "time": 1613762687145
              }
            },
            {
              "dataset": "urn:li:dataset:(urn:li:dataPlatform:mssql,dbo.SAMTEC_PART_MASTER_BASE,PROD)",
              "type": "TRANSFORMED",
              "auditStamp": {
                "actor": "urn:li:corpuser:enterprisearchitect",
                "time": 1613762687145
              }
            },
            {
              "dataset": "urn:li:dataset:(urn:li:dataPlatform:mssql,ShopFloorControl.MaterialRequirement_base,PROD)",
              "type": "TRANSFORMED",
              "auditStamp": {
                "actor": "urn:li:corpuser:enterprisearchitect",
                "time": 1613762687145
              }
            }
          ]
        }
      }
    ]
  }
}
b
thank you! this is great. Let me take a look
👍 1
c
Hello, John, wondering if there is any update on this issue?
b
Couple things I see... First of all, are these datasets all present within DH? Eg additional metadata about them? If not, I think you've stumbled across a bug that I've recently raised a PR to fix: https://github.com/linkedin/datahub/pull/2146
Copy code
[{"dataset":"urn:li:dataset:(urn:li:dataPlatform:mssql,Operations.BillOfMaterial_Base,PROD)","type":"TRANSFORMED","auditStamp":{"actor":"urn:li:corpuser:enterprisearchitect","time":1613762687145}},{"dataset":"urn:li:dataset:(urn:li:dataPlatform:mssql,Operations.Components_Base,PROD)","type":"TRANSFORMED","auditStamp":{"actor":"urn:li:corpuser:enterprisearchitect","time":1613762687145}},{"dataset":"urn:li:dataset:(urn:li:dataPlatform:mssql,dbo.INV_MATERIAL_BASE,PROD)","type":"TRANSFORMED","auditStamp":{"actor":"urn:li:corpuser:enterprisearchitect","time":1613762687145}},{"dataset":"urn:li:dataset:(urn:li:dataPlatform:mssql,dbo.SAMTEC_PART_MASTER_BASE,PROD)","type":"TRANSFORMED","auditStamp":{"actor":"urn:li:corpuser:enterprisearchitect","time":1613762687145}},{"dataset":"urn:li:dataset:(urn:li:dataPlatform:mssql,ShopFloorControl.MaterialRequirement_base,PROD)","type":"TRANSFORMED","auditStamp":{"actor":"urn:li:corpuser:enterprisearchitect","time":1613762687145}}]}}]}}
Second, is the dataPlatform "mssql" present in your DataPlatformInfo.json file? https://github.com/linkedin/datahub/blob/master/gms/impl/src/main/resources/DataPlatformInfo.json If not, please add an entry in your file and re-deploy. In the meantime, I'll create a PR to add it to the default list! We validate against this set of dataPlatforms on read... if it's
I think one or both of these issues would be impacting you. Let me try to get these changes in today
c
thanks for checking on it.
Copy code
1. Yes, all datasets in upstream lineage are present in DataHub. 
2. mssql not in dataplatformInfo.json, although all mssql datasets are loading fine in both UI.
I will try to get latest codes/docker images early next week.
b
okay we've made many improvements to react recently (still in beta) so it's hard to pinpoint
👍 1
4 Views
Open in Slack
PreviousNext
Powered by