Hi all,I want to package pypi myself,I found that ...
# all-things-deployment
f
Hi all,I want to package pypi myself,I found that the metadata module is missing after packing。This module should be generated automatically. I don't know how to generate it @dazzling-judge-80093
Copy code
../gradlew :metadata-ingestion:installDev
Copy code
Failed to build python-ldap orderedset sasl3
ERROR: Could not build wheels for python-ldap, which is required to install pyproject.toml-based projects

> Task :metadata-ingestion:installDev FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':metadata-ingestion:installDev'.
> Process 'command 'bash'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/6.9.2/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD FAILED in 4m 19s
26 actionable tasks: 9 executed, 17 up-to-date
b
Ping on @gray-shoe-75895 -- hoping we can help with the packaging question 🙂
g
@full-chef-85630 Based on the logs, it looks like it was unable to build/install some python dependencies. What OS are you using?
Additionally, I’m curious why you’d like to package for pypi yourself? We have a few hooks built into our existing platform that should make certain extensions easy to build
f
Hi @gray-shoe-75895
Copy code
[root@VM-4-16-centos ~]# uname -a
Linux VM-4-16-centos 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Copy code
[root@VM-4-16-centos ~]# python3 --version
Python 3.8.8
Copy code
[root@VM-4-16-centos ~]# gradle -v

------------------------------------------------------------
Gradle 7.5.1
------------------------------------------------------------

Build time:   2022-08-05 21:17:56 UTC
Revision:     d1daa0cbf1a0103000b71484e1dbfe096e095918

Kotlin:       1.6.21
Groovy:       3.0.10
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          1.8.0_312 (Red Hat, Inc. 25.312-b07)
OS:           Linux 4.18.0-348.7.1.el8_5.x86_64 amd64

[root@VM-4-16-centos ~]#
Because the version we are using now is 0.8.41, there is a bug in this version, and we need to repack it
g
Could you try running
../gradlew :metadata-ingestion:codegen
? It seems like enough of your python setup completed that the codegen should work
f
not error , but /src/datahub/metadata/schema not generate
No problem, it’s all right
running , why not successful
Copy code
../gradlew :metadata-ingestion:installDev
g
The issue was that installDev was failing and hence the build process was never getting to the codegen part. Running the codegen step directly avoided that issue
I am curious what the motivation is behind building a custom pypi package - do you have custom sources/transformers, or have you made other modifications to the framework?