noticed that whenever i have a policy with an entity filter (for instance, a new metadata policy applicable to notebook entities only) and i try to query the new policy using something like

curr_policy = graph.get_aspect_v2(
        entity_urn=policy_urn,
        aspect="dataHubPolicyInfo",
        aspect_type=DataHubPolicyInfoClass,
)

I always get the error message:

File "/home/*redacted*/datahub/*redacted*/policy.py", line 54, in <module>
    curr_policy = graph.get_aspect_v2(
  File "/home/*redacted*/datahub/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 171, in get_aspect_v2
    return aspect_type.from_obj(post_json_obj)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/dict_wrapper.py", line 41, in from_obj
    return conv.from_json_object(obj, cls.RECORD_SCHEMA)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/avrojson.py", line 104, in from_json_object
    return self._generic_from_json(json_obj, writers_schema, readers_schema)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/avrojson.py", line 257, in _generic_from_json
    result = self._record_from_json(json_obj, writers_schema, readers_schema)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/avrojson.py", line 345, in _record_from_json
    field_value = self._generic_from_json(json_obj[field.name], writers_field.type, field.type)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/avrojson.py", line 255, in _generic_from_json
    result = self._union_from_json(json_obj, writers_schema, readers_schema)
  File "/home/*redacted*/miniconda3/envs/*redacted*/lib/python3.9/site-packages/avrogen/avrojson.py", line 314, in _union_from_json
    raise schema.AvroException('Datum union type not in schema: %s', value_type)
avro.schema.AvroException: ('Datum union type not in schema: %s', 'filter')

any idea what causes this? What is weird is that it can be overcame by going to the policy, adding another entity, saving it, and then undoing it again from UI. Then querying the policy again will not have the same error. Almost like there was something missing the first time round when it was created... I was trying to query all my policies and store them as a json file (backup)

This might indicate a bug in our codegen classes on the python side - looking into it

Thanks Harshal!