Hi team, I have below sample druid dim column fetc...
# general
v
Hi team, I have below sample druid dim column fetch but values mentioned in
dimensionOverrideMap
are not rendered (i.e, though the lookup returned "null" (as string) it is not replaced with "" (empty) string. Any issue in specifying extractionFn?
Copy code
{
  "type": "extraction",
  "dimension": "acc_id",
  "outputName": "AgencyName",
  "outputType": "STRING",
  "extractionFn": {
    "type": "registeredLookup",
    "lookup": "agency_lookup",
    "retainMissingValue": false,
    "replaceMissingValueWith": "null",
    "injective": false,
    "optimize": true,
    "valueColumn": "business_name",
    "dimensionOverrideMap": {
      "null": "",
      "-1": "N/A",
      "-3": "N/A"
    },
    "useQueryLevelCache": false
  }
}
t
Based on the information retrieved, it seems that you are using a Druid configuration with an extraction function, specifically a registered lookup, and are encountering issues with the rendering of values from the
dimensionOverrideMap
. Here are a few points to consider regarding your current configuration: 1. retainMissingValue: Since you have set this property to
false
, it means that any missing values will not be retained. This is useful if you want to clean up null or missing values but might lead to unexpected behavior if combined with the
replaceMissingValueWith
option. 2. replaceMissingValueWith: You have specified
"null"
(as a string) to replace missing values. If your expectation is to replace these with an empty string
""
, you should set this parameter to
""
instead. 3. dimensionOverrideMap: You are defining how specific values should be overridden. However, the key
"null"
in JavaScript (JSON compatible) isn't quite accurate for distinguishing between actual null values and the string "null". Make sure the values in this map correspond precisely to how values are returned from the lookup. 4. lookup returned values: If the lookup returns an actual null (which is different from the string
"null"
), it will not trigger replacements as specified in your
dimensionOverrideMap
. Ensure that your lookup mapping correctly handles these cases. 5. debugging: For effective debugging, evaluate the outputs of your lookups. Check whether the values being returned match what you expect. You might also want to confirm that the lookup table being referenced (
agency_lookup
) is configured correctly and holds the correct mappings. Make necessary adjustments based on the feedback above and consider testing your extraction function configuration using various input cases to ensure that it behaves as intended. Sources: Extraction Function Details: filters.md (internal document) Druid SQL Compatibility Changes: upgrade-notes.md (internal document) Data Types and Null Handling: sql-data-types.md (internal document)