https://datahubproject.io logo
Join Slack
Powered by
Hi team, the `:metadata-ingestion:lint` task faile...
# ingestion
b
Hi team, the 
:metadata-ingestion:lint
task failed due to lint errors when running 
mypy
command. There are 72 errors, listed in thread. A suggestion - do you think if lint check can be enforced when there are PRs to 
metadata-ingestion
for better QA? 😅
mypy src/ tests/ examples/
Copy code
src/datahub/configuration/common.py:103: error: Incompatible types in assignment (expression has type "bool", variable has type "Optional[str]")
src/datahub/ingestion/source/glue_profiling_config.py:7: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:11: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:15: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:19: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:23: error: Incompatible types in assignment (expression has type "None", variable has type "int")
src/datahub/ingestion/source/glue_profiling_config.py:27: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:31: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:35: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:39: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:43: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/glue_profiling_config.py:47: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/mode.py:71: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/mode.py:72: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/metabase.py:51: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/metabase.py:52: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/ldap.py:100: error: Incompatible types in assignment (expression has type "None", variable has type "List[str]")
src/datahub/ingestion/source_config/sql/snowflake.py:84: error: Incompatible types in assignment (expression has type "None", variable has type "SecretStr")
src/datahub/ingestion/source/superset.py:62: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/dbt.py:156: error: Incompatible types in assignment (expression has type "str", variable has type "EmitDirective")
src/datahub/ingestion/source/dbt.py:159: error: Incompatible types in assignment (expression has type "str", variable has type "EmitDirective")
src/datahub/ingestion/source/dbt.py:162: error: Incompatible types in assignment (expression has type "str", variable has type "EmitDirective")
src/datahub/ingestion/source/dbt.py:165: error: Incompatible types in assignment (expression has type "str", variable has type "EmitDirective")
src/datahub/ingestion/source/dbt.py:169: error: Incompatible types in assignment (expression has type "str", variable has type "EmitDirective")
src/datahub/ingestion/source/usage/starburst_trino_usage.py:65: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/usage/starburst_trino_usage.py:69: error: Incompatible types in assignment (expression has type "None", variable has type "TrinoConnectorInfo")
src/datahub/ingestion/source/usage/starburst_trino_usage.py:76: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/usage/starburst_trino_usage.py:80: error: Incompatible types in assignment (expression has type "None", variable has type "datetime")
src/datahub/ingestion/source/usage/starburst_trino_usage.py:81: error: Incompatible types in assignment (expression has type "None", variable has type "datetime")
src/datahub/ingestion/source/delta_lake/config.py:22: error: Incompatible types in assignment (expression has type "None", variable has type "AwsSourceConfig")
src/datahub/ingestion/source/delta_lake/config.py:43: error: Incompatible types in assignment (expression has type "None", variable has type "str")
src/datahub/ingestion/source/aws/glue.py:126: error: Incompatible types in assignment (expression has type "None", variable has type "GlueProfilingConfig")
src/datahub/ingestion/source/usage/redshift_usage.py:137: error: Incompatible types in assignment (expression has type "None", variable has type "str")
tests/unit/test_iceberg.py:31: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:31: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:31: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:31: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:36: error: Missing named argument "localfs" for "IcebergSourceConfig"
tests/unit/test_iceberg.py:36: error: Missing named argument "group_ownership_property" for "IcebergSourceConfig"
tests/unit/test_iceberg.py:62: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:62: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:62: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:62: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:62: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:69: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:69: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:69: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:69: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:76: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:76: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:76: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:76: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:83: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:83: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:93: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:93: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:93: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:102: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:102: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:102: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:111: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:111: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:111: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:133: error: Missing named argument "adls" for "IcebergSourceConfig"
tests/unit/test_iceberg.py:133: error: Missing named argument "localfs" for "IcebergSourceConfig"
tests/unit/test_iceberg.py:133: error: Missing named argument "group_ownership_property" for "IcebergSourceConfig"
tests/unit/test_iceberg.py:142: error: Missing named argument "account_key" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:142: error: Missing named argument "sas_token" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:142: error: Missing named argument "client_secret" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:142: error: Missing named argument "client_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:142: error: Missing named argument "tenant_id" for "AdlsSourceConfig"
tests/unit/test_iceberg.py:147: error: Missing named argument "group_ownership_property" for "IcebergSourceConfig"
Found 72 errors in 13 files (checked 517 source files)
h
@boundless-student-48844 I believe, the lint checks are already enforced for PRs on metadata-ingestion. Are you still facing the issue ? Which python version are you using ?
a
Does anyone have resolved this ?
h
Hi @acceptable-judge-21659 which python version are you using ? lint is already validated against metadata ingestion for python 3.7 and python 3.10 and is success.
a
Damn... I am on python 3.9...
I had other build problems on python 3.10 so I reverted to 3.9. What about 3.7 ? I can build all the app on 3.7 ?
b
It occurs to me on Python 3.10 too. If you read the mypy errors above, actually those are valid errors
but somehow it can’t be replayed in every local environment, it doesn’t occur on my teammate’s altho he’s running 3.10 too
9 Views
Open in Slack
PreviousNext
Powered by